agora inbox for pgsql-hackers@postgresql.orghelp / color / mirror / Atom feed
[PATCH v8 6/7] Row pattern recognition patch (tests). 408+ messages / 2 participants [nested] [flat]
* [PATCH v8 6/7] Row pattern recognition patch (tests). @ 2023-09-25 05:01 Tatsuo Ishii <ishii@postgresql.org> 0 siblings, 0 replies; 408+ messages in thread From: Tatsuo Ishii @ 2023-09-25 05:01 UTC (permalink / raw) --- src/test/regress/expected/rpr.out | 594 +++++++++++++++++++++++++++++ src/test/regress/parallel_schedule | 2 +- src/test/regress/sql/rpr.sql | 292 ++++++++++++++ 3 files changed, 887 insertions(+), 1 deletion(-) create mode 100644 src/test/regress/expected/rpr.out create mode 100644 src/test/regress/sql/rpr.sql diff --git a/src/test/regress/expected/rpr.out b/src/test/regress/expected/rpr.out new file mode 100644 index 0000000000..3e7ad943c9 --- /dev/null +++ b/src/test/regress/expected/rpr.out @@ -0,0 +1,594 @@ +-- +-- Test for row pattern definition clause +-- +CREATE TEMP TABLE stock ( + company TEXT, + tdate DATE, + price INTEGER +); +INSERT INTO stock VALUES ('company1', '2023-07-01', 100); +INSERT INTO stock VALUES ('company1', '2023-07-02', 200); +INSERT INTO stock VALUES ('company1', '2023-07-03', 150); +INSERT INTO stock VALUES ('company1', '2023-07-04', 140); +INSERT INTO stock VALUES ('company1', '2023-07-05', 150); +INSERT INTO stock VALUES ('company1', '2023-07-06', 90); +INSERT INTO stock VALUES ('company1', '2023-07-07', 110); +INSERT INTO stock VALUES ('company1', '2023-07-08', 130); +INSERT INTO stock VALUES ('company1', '2023-07-09', 120); +INSERT INTO stock VALUES ('company1', '2023-07-10', 130); +INSERT INTO stock VALUES ('company2', '2023-07-01', 50); +INSERT INTO stock VALUES ('company2', '2023-07-02', 2000); +INSERT INTO stock VALUES ('company2', '2023-07-03', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-04', 1400); +INSERT INTO stock VALUES ('company2', '2023-07-05', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-06', 60); +INSERT INTO stock VALUES ('company2', '2023-07-07', 1100); +INSERT INTO stock VALUES ('company2', '2023-07-08', 1300); +INSERT INTO stock VALUES ('company2', '2023-07-09', 1200); +INSERT INTO stock VALUES ('company2', '2023-07-10', 1300); +SELECT * FROM stock; + company | tdate | price +----------+------------+------- + company1 | 07-01-2023 | 100 + company1 | 07-02-2023 | 200 + company1 | 07-03-2023 | 150 + company1 | 07-04-2023 | 140 + company1 | 07-05-2023 | 150 + company1 | 07-06-2023 | 90 + company1 | 07-07-2023 | 110 + company1 | 07-08-2023 | 130 + company1 | 07-09-2023 | 120 + company1 | 07-10-2023 | 130 + company2 | 07-01-2023 | 50 + company2 | 07-02-2023 | 2000 + company2 | 07-03-2023 | 1500 + company2 | 07-04-2023 | 1400 + company2 | 07-05-2023 | 1500 + company2 | 07-06-2023 | 60 + company2 | 07-07-2023 | 1100 + company2 | 07-08-2023 | 1300 + company2 | 07-09-2023 | 1200 + company2 | 07-10-2023 | 1300 +(20 rows) + +-- basic test using PREV +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | | | + company1 | 07-06-2023 | 90 | 90 | 120 | 07-07-2023 + company1 | 07-07-2023 | 110 | | | + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | | | + company2 | 07-06-2023 | 60 | 60 | 1200 | 07-07-2023 + company2 | 07-07-2023 | 1100 | | | + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +-- last_value() should remain consistent +SELECT company, tdate, price, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | last_value +----------+------------+-------+------------ + company1 | 07-01-2023 | 100 | 140 + company1 | 07-02-2023 | 200 | + company1 | 07-03-2023 | 150 | + company1 | 07-04-2023 | 140 | + company1 | 07-05-2023 | 150 | + company1 | 07-06-2023 | 90 | 120 + company1 | 07-07-2023 | 110 | + company1 | 07-08-2023 | 130 | + company1 | 07-09-2023 | 120 | + company1 | 07-10-2023 | 130 | + company2 | 07-01-2023 | 50 | 1400 + company2 | 07-02-2023 | 2000 | + company2 | 07-03-2023 | 1500 | + company2 | 07-04-2023 | 1400 | + company2 | 07-05-2023 | 1500 | + company2 | 07-06-2023 | 60 | 1200 + company2 | 07-07-2023 | 1100 | + company2 | 07-08-2023 | 1300 | + company2 | 07-09-2023 | 1200 | + company2 | 07-10-2023 | 1300 | +(20 rows) + +-- omit "START" in DEFINE but it is ok because "START AS TRUE" is +-- implicitly defined. per spec. +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | nth_second +----------+------------+-------+-------------+------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 | 07-02-2023 + company1 | 07-02-2023 | 200 | | | + company1 | 07-03-2023 | 150 | | | + company1 | 07-04-2023 | 140 | | | + company1 | 07-05-2023 | 150 | | | + company1 | 07-06-2023 | 90 | 90 | 120 | 07-07-2023 + company1 | 07-07-2023 | 110 | | | + company1 | 07-08-2023 | 130 | | | + company1 | 07-09-2023 | 120 | | | + company1 | 07-10-2023 | 130 | | | + company2 | 07-01-2023 | 50 | 50 | 1400 | 07-02-2023 + company2 | 07-02-2023 | 2000 | | | + company2 | 07-03-2023 | 1500 | | | + company2 | 07-04-2023 | 1400 | | | + company2 | 07-05-2023 | 1500 | | | + company2 | 07-06-2023 | 60 | 60 | 1200 | 07-07-2023 + company2 | 07-07-2023 | 1100 | | | + company2 | 07-08-2023 | 1300 | | | + company2 | 07-09-2023 | 1200 | | | + company2 | 07-10-2023 | 1300 | | | +(20 rows) + +-- the first row start with less than or equal to 100 +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | 90 | 120 + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1400 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | 60 | 1200 + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- second row raises 120% +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price) * 1.2, + DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 140 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1400 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- using NEXT +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 200 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | 140 | 150 + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 110 | 130 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 2000 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | 1400 | 1500 + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 1100 | 1300 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 200 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | 140 | 150 + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 110 | 130 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 2000 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | 1400 | 1500 + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 1100 | 1300 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- match everything +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+) + DEFINE + A AS TRUE +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | 100 | 130 + company1 | 07-02-2023 | 200 | | + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | | + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | 50 | 1300 + company2 | 07-02-2023 | 2000 | | + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | | + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | | + company1 | 07-02-2023 | 200 | 07-02-2023 | 07-05-2023 + company1 | 07-03-2023 | 150 | | + company1 | 07-04-2023 | 140 | | + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 07-07-2023 | 07-10-2023 + company1 | 07-08-2023 | 130 | | + company1 | 07-09-2023 | 120 | | + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | | + company2 | 07-02-2023 | 2000 | 07-02-2023 | 07-05-2023 + company2 | 07-03-2023 | 1500 | | + company2 | 07-04-2023 | 1400 | | + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-10-2023 + company2 | 07-08-2023 | 1300 | | + company2 | 07-09-2023 | 1200 | | + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + company | tdate | price | first_value | last_value +----------+------------+-------+-------------+------------ + company1 | 07-01-2023 | 100 | | + company1 | 07-02-2023 | 200 | 07-02-2023 | 07-05-2023 + company1 | 07-03-2023 | 150 | 07-03-2023 | 07-05-2023 + company1 | 07-04-2023 | 140 | 07-04-2023 | 07-05-2023 + company1 | 07-05-2023 | 150 | | + company1 | 07-06-2023 | 90 | | + company1 | 07-07-2023 | 110 | 07-07-2023 | 07-10-2023 + company1 | 07-08-2023 | 130 | 07-08-2023 | 07-10-2023 + company1 | 07-09-2023 | 120 | 07-09-2023 | 07-10-2023 + company1 | 07-10-2023 | 130 | | + company2 | 07-01-2023 | 50 | | + company2 | 07-02-2023 | 2000 | 07-02-2023 | 07-05-2023 + company2 | 07-03-2023 | 1500 | 07-03-2023 | 07-05-2023 + company2 | 07-04-2023 | 1400 | 07-04-2023 | 07-05-2023 + company2 | 07-05-2023 | 1500 | | + company2 | 07-06-2023 | 60 | | + company2 | 07-07-2023 | 1100 | 07-07-2023 | 07-10-2023 + company2 | 07-08-2023 | 1300 | 07-08-2023 | 07-10-2023 + company2 | 07-09-2023 | 1200 | 07-09-2023 | 07-10-2023 + company2 | 07-10-2023 | 1300 | | +(20 rows) + +-- +-- Aggregates +-- +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP PAST LAST ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | max | min | sum | avg | count +----------+------------+-------+-------------+------------+------+-----+------+-----------------------+------- + company1 | 07-01-2023 | 100 | 100 | 140 | 200 | 100 | 590 | 147.5000000000000000 | 4 + company1 | 07-02-2023 | 200 | | | | | | | + company1 | 07-03-2023 | 150 | | | | | | | + company1 | 07-04-2023 | 140 | | | | | | | + company1 | 07-05-2023 | 150 | | | | | | | 0 + company1 | 07-06-2023 | 90 | 90 | 120 | 130 | 90 | 450 | 112.5000000000000000 | 4 + company1 | 07-07-2023 | 110 | | | | | | | + company1 | 07-08-2023 | 130 | | | | | | | + company1 | 07-09-2023 | 120 | | | | | | | + company1 | 07-10-2023 | 130 | | | | | | | 0 + company2 | 07-01-2023 | 50 | 50 | 1400 | 2000 | 50 | 4950 | 1237.5000000000000000 | 4 + company2 | 07-02-2023 | 2000 | | | | | | | + company2 | 07-03-2023 | 1500 | | | | | | | + company2 | 07-04-2023 | 1400 | | | | | | | + company2 | 07-05-2023 | 1500 | | | | | | | 0 + company2 | 07-06-2023 | 60 | 60 | 1200 | 1300 | 60 | 3660 | 915.0000000000000000 | 4 + company2 | 07-07-2023 | 1100 | | | | | | | + company2 | 07-08-2023 | 1300 | | | | | | | + company2 | 07-09-2023 | 1200 | | | | | | | + company2 | 07-10-2023 | 1300 | | | | | | | 0 +(20 rows) + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP TO NEXT ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + company | tdate | price | first_value | last_value | max | min | sum | avg | count +----------+------------+-------+-------------+------------+------+------+------+-----------------------+------- + company1 | 07-01-2023 | 100 | 100 | 140 | 200 | 100 | 590 | 147.5000000000000000 | 4 + company1 | 07-02-2023 | 200 | | | | | | | 0 + company1 | 07-03-2023 | 150 | | | | | | | 0 + company1 | 07-04-2023 | 140 | 140 | 90 | 150 | 90 | 380 | 126.6666666666666667 | 3 + company1 | 07-05-2023 | 150 | | | | | | | 0 + company1 | 07-06-2023 | 90 | 90 | 120 | 130 | 90 | 450 | 112.5000000000000000 | 4 + company1 | 07-07-2023 | 110 | 110 | 120 | 130 | 110 | 360 | 120.0000000000000000 | 3 + company1 | 07-08-2023 | 130 | | | | | | | 0 + company1 | 07-09-2023 | 120 | | | | | | | 0 + company1 | 07-10-2023 | 130 | | | | | | | 0 + company2 | 07-01-2023 | 50 | 50 | 1400 | 2000 | 50 | 4950 | 1237.5000000000000000 | 4 + company2 | 07-02-2023 | 2000 | | | | | | | 0 + company2 | 07-03-2023 | 1500 | | | | | | | 0 + company2 | 07-04-2023 | 1400 | 1400 | 60 | 1500 | 60 | 2960 | 986.6666666666666667 | 3 + company2 | 07-05-2023 | 1500 | | | | | | | 0 + company2 | 07-06-2023 | 60 | 60 | 1200 | 1300 | 60 | 3660 | 915.0000000000000000 | 4 + company2 | 07-07-2023 | 1100 | 1100 | 1200 | 1300 | 1100 | 3600 | 1200.0000000000000000 | 3 + company2 | 07-08-2023 | 1300 | | | | | | | 0 + company2 | 07-09-2023 | 1200 | | | | | | | 0 + company2 | 07-10-2023 | 1300 | | | | | | | 0 +(20 rows) + +-- +-- Error cases +-- +-- row pattern definition variable name must not appear more than once +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + ORDER BY tdate + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price), + UP AS price > PREV(price) +); +ERROR: syntax error at or near "ORDER" +LINE 6: ORDER BY tdate + ^ +-- pattern variable name must appear in DEFINE +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+ END) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +ERROR: syntax error at or near "END" +LINE 8: PATTERN (START UP+ DOWN+ END) + ^ +-- FRAME must start at current row when row patttern recognition is used +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +ERROR: FRAME must start at current row when row patttern recognition is used +-- SEEK is not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + SEEK + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +ERROR: SEEK is not supported +LINE 8: SEEK + ^ +HINT: Use INITIAL. diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule index 4df9d8503b..896531002b 100644 --- a/src/test/regress/parallel_schedule +++ b/src/test/regress/parallel_schedule @@ -98,7 +98,7 @@ test: publication subscription # Another group of parallel tests # select_views depends on create_view # ---------- -test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast equivclass +test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast equivclass rpr # ---------- # Another group of parallel tests (JSON related) diff --git a/src/test/regress/sql/rpr.sql b/src/test/regress/sql/rpr.sql new file mode 100644 index 0000000000..846e1af2d7 --- /dev/null +++ b/src/test/regress/sql/rpr.sql @@ -0,0 +1,292 @@ +-- +-- Test for row pattern definition clause +-- + +CREATE TEMP TABLE stock ( + company TEXT, + tdate DATE, + price INTEGER +); +INSERT INTO stock VALUES ('company1', '2023-07-01', 100); +INSERT INTO stock VALUES ('company1', '2023-07-02', 200); +INSERT INTO stock VALUES ('company1', '2023-07-03', 150); +INSERT INTO stock VALUES ('company1', '2023-07-04', 140); +INSERT INTO stock VALUES ('company1', '2023-07-05', 150); +INSERT INTO stock VALUES ('company1', '2023-07-06', 90); +INSERT INTO stock VALUES ('company1', '2023-07-07', 110); +INSERT INTO stock VALUES ('company1', '2023-07-08', 130); +INSERT INTO stock VALUES ('company1', '2023-07-09', 120); +INSERT INTO stock VALUES ('company1', '2023-07-10', 130); +INSERT INTO stock VALUES ('company2', '2023-07-01', 50); +INSERT INTO stock VALUES ('company2', '2023-07-02', 2000); +INSERT INTO stock VALUES ('company2', '2023-07-03', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-04', 1400); +INSERT INTO stock VALUES ('company2', '2023-07-05', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-06', 60); +INSERT INTO stock VALUES ('company2', '2023-07-07', 1100); +INSERT INTO stock VALUES ('company2', '2023-07-08', 1300); +INSERT INTO stock VALUES ('company2', '2023-07-09', 1200); +INSERT INTO stock VALUES ('company2', '2023-07-10', 1300); + +SELECT * FROM stock; + +-- basic test using PREV +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- last_value() should remain consistent +SELECT company, tdate, price, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- omit "START" in DEFINE but it is ok because "START AS TRUE" is +-- implicitly defined. per spec. +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w, + nth_value(tdate, 2) OVER w AS nth_second + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- the first row start with less than or equal to 100 +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- second row raises 120% +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price) * 1.2, + DOWN AS price < PREV(price) +); + +-- using NEXT +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + +-- match everything + +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+) + DEFINE + A AS TRUE +); + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + +-- backtracking with reclassification of rows +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, first_value(tdate) OVER w, last_value(tdate) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (A+ B+) + DEFINE + A AS price > 100, + B AS price > 100 +); + +-- +-- Aggregates +-- + +-- using AFTER MATCH SKIP PAST LAST ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP PAST LAST ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, + first_value(price) OVER w, + last_value(price) OVER w, + max(price) OVER w, + min(price) OVER w, + sum(price) OVER w, + avg(price) OVER w, + count(price) OVER w +FROM stock +WINDOW w AS ( +PARTITION BY company +ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING +AFTER MATCH SKIP TO NEXT ROW +INITIAL +PATTERN (START UP+ DOWN+) +DEFINE +START AS TRUE, +UP AS price > PREV(price), +DOWN AS price < PREV(price) +); + +-- +-- Error cases +-- + +-- row pattern definition variable name must not appear more than once +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + ORDER BY tdate + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price), + UP AS price > PREV(price) +); + +-- pattern variable name must appear in DEFINE +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+ END) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- FRAME must start at current row when row patttern recognition is used +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- SEEK is not supported +SELECT company, tdate, price, first_value(price) OVER w, last_value(price) OVER w + FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + SEEK + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); -- 2.25.1 ----Next_Part(Mon_Sep_25_14_26_30_2023_752)-- Content-Type: Text/X-Patch; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="v8-0007-Allow-to-print-raw-parse-tree.patch" ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
* [PATCH v49 7/7] WIP add max_repack_replication_slots @ 2026-04-01 17:54 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 408+ messages in thread From: Álvaro Herrera @ 2026-04-01 17:54 UTC (permalink / raw) --- src/backend/commands/repack_worker.c | 4 +- src/backend/replication/logical/launcher.c | 2 +- src/backend/replication/logical/slotsync.c | 5 +- src/backend/replication/slot.c | 77 +++++++++++-------- src/backend/replication/slotfuncs.c | 8 +- src/backend/replication/walsender.c | 4 +- src/backend/utils/misc/guc_parameters.dat | 8 ++ src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/replication/slot.h | 6 +- 9 files changed, 74 insertions(+), 42 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index 106b2b60441..00b21ede481 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -228,8 +228,8 @@ repack_setup_logical_decoding(Oid relid) * RS_TEMPORARY so that the slot gets cleaned up on ERROR. */ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid); - ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false, - false); + ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, true, + false, false); EnsureLogicalDecodingEnabled(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index 09964198550..d83125afd0d 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -1575,7 +1575,7 @@ CreateConflictDetectionSlot(void) errmsg("creating replication conflict detection slot")); ReplicationSlotCreate(CONFLICT_DETECTION_SLOT, false, RS_PERSISTENT, false, - false, false); + false, false, false); init_conflict_slot_xmin(); } diff --git a/src/backend/replication/logical/slotsync.c b/src/backend/replication/logical/slotsync.c index e75db69e3f6..1bc7f3f600d 100644 --- a/src/backend/replication/logical/slotsync.c +++ b/src/backend/replication/logical/slotsync.c @@ -425,7 +425,7 @@ get_local_synced_slots(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -814,6 +814,7 @@ synchronize_one_slot(RemoteSlot *remote_slot, Oid remote_dbid, */ ReplicationSlotCreate(remote_slot->name, true, RS_TEMPORARY, remote_slot->two_phase, + false, remote_slot->failover, true); @@ -1691,7 +1692,7 @@ update_synced_slots_inactive_since(void) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index a9092fc2382..fe6bfeba25e 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -151,6 +151,9 @@ ReplicationSlot *MyReplicationSlot = NULL; /* GUC variables */ int max_replication_slots = 10; /* the maximum number of replication * slots */ +int max_repack_replication_slots = 1; /* the maximum number of slots + * for REPACK */ +int TotalMaxReplicationSlots = 0; /* sum of both */ /* * Invalidate replication slots that have remained idle longer than this @@ -190,12 +193,14 @@ ReplicationSlotsShmemSize(void) { Size size = 0; - if (max_replication_slots == 0) + TotalMaxReplicationSlots = max_replication_slots + max_repack_replication_slots; + + if (TotalMaxReplicationSlots == 0) return size; size = offsetof(ReplicationSlotCtlData, replication_slots); size = add_size(size, - mul_size(max_replication_slots, sizeof(ReplicationSlot))); + mul_size(TotalMaxReplicationSlots, sizeof(ReplicationSlot))); return size; } @@ -208,7 +213,7 @@ ReplicationSlotsShmemInit(void) { bool found; - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return; ReplicationSlotCtl = (ReplicationSlotCtlData *) @@ -222,7 +227,7 @@ ReplicationSlotsShmemInit(void) /* First time through, so initialize */ MemSet(ReplicationSlotCtl, 0, ReplicationSlotsShmemSize()); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[i]; @@ -372,6 +377,7 @@ IsSlotForConflictCheck(const char *name) * db_specific: logical decoding is db specific; if the slot is going to * be used for that pass true, otherwise false. * two_phase: If enabled, allows decoding of prepared transactions. + * repack: If true, use a slot from the pool for REPACK. * failover: If enabled, allows the slot to be synced to standbys so * that logical replication can be resumed after failover. * synced: True if the slot is synchronized from the primary server. @@ -379,10 +385,11 @@ IsSlotForConflictCheck(const char *name) void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, bool synced) + bool two_phase, bool repack, bool failover, bool synced) { ReplicationSlot *slot = NULL; - int i; + int startpoint, + endpoint; Assert(MyReplicationSlot == NULL); @@ -431,12 +438,16 @@ ReplicationSlotCreate(const char *name, bool db_specific, LWLockAcquire(ReplicationSlotAllocationLock, LW_EXCLUSIVE); /* - * Check for name collision, and identify an allocatable slot. We need to - * hold ReplicationSlotControlLock in shared mode for this, so that nobody - * else can change the in_use flags while we're looking at them. + * Check for name collision (across the whole array), and identify an + * allocatable slot (in the array slice specific to our current use case: + * either general, or REPACK only). We need to hold + * ReplicationSlotControlLock in shared mode for this, so that nobody else + * can change the in_use flags while we're looking at them. */ LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + startpoint = repack ? max_replication_slots : 0; + endpoint = repack ? TotalMaxReplicationSlots : max_replication_slots; + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -444,7 +455,9 @@ ReplicationSlotCreate(const char *name, bool db_specific, ereport(ERROR, (errcode(ERRCODE_DUPLICATE_OBJECT), errmsg("replication slot \"%s\" already exists", name))); - if (!s->in_use && slot == NULL) + + if (i >= startpoint && i < endpoint && + !s->in_use && slot == NULL) slot = s; } LWLockRelease(ReplicationSlotControlLock); @@ -547,7 +560,7 @@ SearchNamedReplicationSlot(const char *name, bool need_lock) if (need_lock) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -575,7 +588,7 @@ int ReplicationSlotIndex(ReplicationSlot *slot) { Assert(slot >= ReplicationSlotCtl->replication_slots && - slot < ReplicationSlotCtl->replication_slots + max_replication_slots); + slot < ReplicationSlotCtl->replication_slots + TotalMaxReplicationSlots); return slot - ReplicationSlotCtl->replication_slots; } @@ -869,7 +882,7 @@ ReplicationSlotCleanup(bool synced_only) restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; @@ -1251,7 +1264,7 @@ ReplicationSlotsComputeRequiredXmin(bool already_locked) if (!already_locked) LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; TransactionId effective_xmin; @@ -1306,7 +1319,7 @@ ReplicationSlotsComputeRequiredLSN(void) Assert(ReplicationSlotCtl != NULL); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; XLogRecPtr restart_lsn; @@ -1373,12 +1386,12 @@ ReplicationSlotsComputeLogicalRestartLSN(void) XLogRecPtr result = InvalidXLogRecPtr; int i; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return InvalidXLogRecPtr; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; XLogRecPtr restart_lsn; @@ -1453,11 +1466,11 @@ ReplicationSlotsCountDBSlots(Oid dboid, int *nslots, int *nactive) *nslots = *nactive = 0; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; @@ -1514,13 +1527,13 @@ ReplicationSlotsDropDBSlots(Oid dboid) bool found_valid_logicalslot; bool dropped = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; char *slotname; @@ -1617,11 +1630,11 @@ CheckLogicalSlotExists(void) { bool found = false; - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s; bool invalidated; @@ -1662,7 +1675,10 @@ CheckSlotRequirements(void) * needs the same check. */ - if (max_replication_slots == 0) + /* FIXME how to relax this for repack in a way that doesn't mess everything + * up? + */ + if (TotalMaxReplicationSlots == 0) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("replication slots can only be used if \"max_replication_slots\" > 0"))); @@ -2216,7 +2232,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, Assert(!(possible_causes & RS_INVAL_WAL_REMOVED) || oldestSegno > 0); Assert(possible_causes != RS_INVAL_NONE); - if (max_replication_slots == 0) + if (TotalMaxReplicationSlots == 0) return invalidated; XLogSegNoOffsetToRecPtr(oldestSegno, 0, wal_segment_size, oldestLSN); @@ -2224,7 +2240,7 @@ InvalidateObsoleteReplicationSlots(uint32 possible_causes, restart: found_valid_logicalslot = false; LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; bool released_lock = false; @@ -2329,7 +2345,7 @@ CheckPointReplicationSlots(bool is_shutdown) */ LWLockAcquire(ReplicationSlotAllocationLock, LW_SHARED); - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; char path[MAXPGPATH]; @@ -2430,7 +2446,7 @@ StartupReplicationSlots(void) FreeDir(replication_dir); /* currently no slots exist, we're done. */ - if (max_replication_slots <= 0) + if (TotalMaxReplicationSlots <= 0) return; /* Now that we have recovered all the data, compute replication xmin */ @@ -2860,7 +2876,7 @@ RestoreSlotFromDisk(const char *name) errhint("Change \"wal_level\" to be \"replica\" or higher."))); /* nothing can be active yet, don't lock anything */ - for (i = 0; i < max_replication_slots; i++) + for (i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *slot; @@ -2902,6 +2918,7 @@ RestoreSlotFromDisk(const char *name) break; } + /* XXX might be misleading if the slots previously in use were REPACK. */ if (!restored) ereport(FATAL, (errmsg("too many replication slots active before shutdown"), diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c index 9f5e4f998fe..cdb4dbd87c9 100644 --- a/src/backend/replication/slotfuncs.c +++ b/src/backend/replication/slotfuncs.c @@ -53,7 +53,7 @@ create_physical_replication_slot(char *name, bool immediately_reserve, /* acquire replication slot, this will check for conflicting names */ ReplicationSlotCreate(name, false, temporary ? RS_TEMPORARY : RS_PERSISTENT, false, - false, false); + false, false, false); if (immediately_reserve) { @@ -146,7 +146,7 @@ create_logical_replication_slot(char *name, char *plugin, */ ReplicationSlotCreate(name, true, temporary ? RS_TEMPORARY : RS_EPHEMERAL, two_phase, - failover, false); + false, failover, false); /* * Ensure the logical decoding is enabled before initializing the logical @@ -270,7 +270,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) currlsn = GetXLogWriteRecPtr(); LWLockAcquire(ReplicationSlotControlLock, LW_SHARED); - for (slotno = 0; slotno < max_replication_slots; slotno++) + for (slotno = 0; slotno < TotalMaxReplicationSlots; slotno++) { ReplicationSlot *slot = &ReplicationSlotCtl->replication_slots[slotno]; ReplicationSlot slot_contents; @@ -665,7 +665,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot) * managed to create the new slot, we advance the new slot's restart_lsn * to the source slot's updated restart_lsn the second time we lock it. */ - for (int i = 0; i < max_replication_slots; i++) + for (int i = 0; i < TotalMaxReplicationSlots; i++) { ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fbbe09135bf..caea3ed1257 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1220,7 +1220,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) { ReplicationSlotCreate(cmd->slotname, false, cmd->temporary ? RS_TEMPORARY : RS_PERSISTENT, - false, false, false); + false, false, false, false); if (reserve_wal) { @@ -1251,7 +1251,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd) */ ReplicationSlotCreate(cmd->slotname, true, cmd->temporary ? RS_TEMPORARY : RS_EPHEMERAL, - two_phase, failover, false); + two_phase, false, failover, false); /* * Do options check early so that we can bail before calling the diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 0a862693fcd..d12a3bd7bdf 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -2070,6 +2070,14 @@ max => 'MAX_BACKENDS', }, +{ name => 'max_repack_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', + short_desc => 'Sets the maximum number of replication slots for use by REPACK.', + variable => 'max_repack_replication_slots', + boot_val => '1', + min => '0', + max => 'MAX_BACKENDS /* XXX? */', +}, + /* see max_wal_senders */ { name => 'max_replication_slots', type => 'int', context => 'PGC_POSTMASTER', group => 'REPLICATION_SENDING', short_desc => 'Sets the maximum number of simultaneously defined replication slots.', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index cf15597385b..15014f37753 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -348,6 +348,8 @@ # (change requires restart) #max_replication_slots = 10 # max number of replication slots # (change requires restart) +#max_repack_replication_slots = 1 # max number of replication slots for REPACK + # (change requires restart) #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #idle_replication_slot_timeout = 0 # in seconds; 0 disables diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h index 4b4709f6e2c..88953576894 100644 --- a/src/include/replication/slot.h +++ b/src/include/replication/slot.h @@ -324,9 +324,13 @@ extern PGDLLIMPORT ReplicationSlot *MyReplicationSlot; /* GUCs */ extern PGDLLIMPORT int max_replication_slots; +extern PGDLLIMPORT int max_repack_replication_slots; extern PGDLLIMPORT char *synchronized_standby_slots; extern PGDLLIMPORT int idle_replication_slot_timeout_secs; +/* only for slotfuncs.c, slotsync.c etc */ +extern int TotalMaxReplicationSlots; + /* shmem initialization functions */ extern Size ReplicationSlotsShmemSize(void); extern void ReplicationSlotsShmemInit(void); @@ -334,7 +338,7 @@ extern void ReplicationSlotsShmemInit(void); /* management of individual slots */ extern void ReplicationSlotCreate(const char *name, bool db_specific, ReplicationSlotPersistency persistency, - bool two_phase, bool failover, + bool two_phase, bool repack, bool failover, bool synced); extern void ReplicationSlotPersist(void); extern void ReplicationSlotDrop(const char *name, bool nowait); -- 2.47.3 --3n3vqr5y2jhknrqj-- ^ permalink raw reply [nested|flat] 408+ messages in thread
end of thread, other threads:[~2026-04-01 17:54 UTC | newest] Thread overview: 408+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2023-09-25 05:01 [PATCH v8 6/7] Row pattern recognition patch (tests). Tatsuo Ishii <ishii@postgresql.org> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de> 2026-04-01 17:54 [PATCH v49 7/7] WIP add max_repack_replication_slots Álvaro Herrera <alvherre@kurilemu.de>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox