public inbox for [email protected]
help / color / mirror / Atom feedRe: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
32+ messages / 14 participants
[nested] [flat]
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-11-20 06:26 Guillaume Lelarge <[email protected]>
0 siblings, 1 reply; 32+ messages in thread
From: Guillaume Lelarge @ 2024-11-20 06:26 UTC (permalink / raw)
To: Greg Sabino Mullane <[email protected]>; +Cc: Robert Haas <[email protected]>; Michael Christofides <[email protected]>; David Rowley <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
Le mer. 20 nov. 2024 à 01:13, Greg Sabino Mullane <[email protected]> a
écrit :
> I'm with Robert on this one. I'm a grudging +1 to buffers defaulting on,
> and a strong -1 to all the other proposals. Guillaume, the patch looks
> pretty good.
>
Thanks :)
> I would like to see some of the example output have more than just "shared
> hit" and "read" though: let's throw some "dirtied" and "written" in
> there as well.
>
>
OK, but I'm not sure which example I should pick to add dirtied and written
shared buffers. It looks kinda artificial. Should I choose one randomly?
It defaults to <literal>TRUE</literal> when <literal>ANALYZE</literal> is
>> also enabled. Otherwise, it defaults to <literal>FALSE</literal>.
>
>
> Is that second sentence really needed? Because "BUFFERS ON" will never be
> needed anymore (save as a no-op to allow the same explain queries to run
> cross-version), and BUFFERS OFF outside of analyze is meaningless.
>
>
Done in v3 with David's idea.
--
Guillaume.
Attachments:
[text/x-patch] v3-0001-Enable-BUFFERS-by-default-with-EXPLAIN-ANALYZE.patch (77.0K, ../../CAECtzeUwpK5gNWQqoK4ZjwXANtdE2fWirJyaBDqkytPzfWZqzg@mail.gmail.com/3-v3-0001-Enable-BUFFERS-by-default-with-EXPLAIN-ANALYZE.patch)
download | inline diff:
From 5b815f71e874522bcc5947a29623328fdc76104b Mon Sep 17 00:00:00 2001
From: Guillaume Lelarge <[email protected]>
Date: Tue, 12 Nov 2024 21:59:14 +0100
Subject: [PATCH v3] Enable BUFFERS by default with EXPLAIN ANALYZE
This automatically enables the BUFFER option when ANALYZE is used with
EXPLAIN. The BUFFER option stays disabled if ANALYZE isn't used.
The auto_explain extension gets the same treatment.
---
contrib/auto_explain/auto_explain.c | 4 +-
doc/src/sgml/auto-explain.sgml | 2 +-
doc/src/sgml/perform.sgml | 42 ++++++---
doc/src/sgml/ref/explain.sgml | 6 +-
src/backend/commands/explain.c | 7 ++
src/test/regress/expected/brin_multi.out | 18 ++--
src/test/regress/expected/explain.out | 36 ++++++-
.../regress/expected/incremental_sort.out | 4 +-
src/test/regress/expected/memoize.out | 2 +-
src/test/regress/expected/merge.out | 2 +-
src/test/regress/expected/partition_prune.out | 94 +++++++++----------
src/test/regress/expected/select.out | 2 +-
src/test/regress/expected/select_into.out | 4 +-
src/test/regress/expected/select_parallel.out | 6 +-
src/test/regress/expected/subselect.out | 2 +-
src/test/regress/expected/tidscan.out | 6 +-
src/test/regress/sql/brin_multi.sql | 18 ++--
src/test/regress/sql/explain.sql | 7 +-
src/test/regress/sql/incremental_sort.sql | 4 +-
src/test/regress/sql/memoize.sql | 2 +-
src/test/regress/sql/merge.sql | 2 +-
src/test/regress/sql/partition_prune.sql | 94 +++++++++----------
src/test/regress/sql/select.sql | 2 +-
src/test/regress/sql/select_into.sql | 4 +-
src/test/regress/sql/select_parallel.sql | 6 +-
src/test/regress/sql/subselect.sql | 2 +-
src/test/regress/sql/tidscan.sql | 6 +-
27 files changed, 220 insertions(+), 164 deletions(-)
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 623a674f99..484e009577 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -27,7 +27,7 @@ static int auto_explain_log_min_duration = -1; /* msec or -1 */
static int auto_explain_log_parameter_max_length = -1; /* bytes or -1 */
static bool auto_explain_log_analyze = false;
static bool auto_explain_log_verbose = false;
-static bool auto_explain_log_buffers = false;
+static bool auto_explain_log_buffers = true;
static bool auto_explain_log_wal = false;
static bool auto_explain_log_triggers = false;
static bool auto_explain_log_timing = true;
@@ -152,7 +152,7 @@ _PG_init(void)
"Log buffers usage.",
NULL,
&auto_explain_log_buffers,
- false,
+ true,
PGC_SUSET,
0,
NULL,
diff --git a/doc/src/sgml/auto-explain.sgml b/doc/src/sgml/auto-explain.sgml
index 0c4656ee30..6623d36c99 100644
--- a/doc/src/sgml/auto-explain.sgml
+++ b/doc/src/sgml/auto-explain.sgml
@@ -122,7 +122,7 @@ LOAD 'auto_explain';
equivalent to the <literal>BUFFERS</literal> option of <command>EXPLAIN</command>.
This parameter has no effect
unless <varname>auto_explain.log_analyze</varname> is enabled.
- This parameter is off by default.
+ This parameter is on by default.
Only superusers can change this setting.
</para>
</listitem>
diff --git a/doc/src/sgml/perform.sgml b/doc/src/sgml/perform.sgml
index cd12b9ce48..b353cc77b6 100644
--- a/doc/src/sgml/perform.sgml
+++ b/doc/src/sgml/perform.sgml
@@ -722,13 +722,19 @@ WHERE t1.unique1 < 10 AND t1.unique2 = t2.unique2;
QUERY PLAN
-------------------------------------------------------------------&zwsp;--------------------------------------------------------------
Nested Loop (cost=4.65..118.50 rows=10 width=488) (actual time=0.017..0.051 rows=10 loops=1)
+ Buffers: shared hit=36 read=6
-> Bitmap Heap Scan on tenk1 t1 (cost=4.36..39.38 rows=10 width=244) (actual time=0.009..0.017 rows=10 loops=1)
Recheck Cond: (unique1 < 10)
Heap Blocks: exact=10
+ Buffers: shared hit=12
-> Bitmap Index Scan on tenk1_unique1 (cost=0.00..4.36 rows=10 width=0) (actual time=0.004..0.004 rows=10 loops=1)
Index Cond: (unique1 < 10)
+ Buffers: shared hit=2
-> Index Scan using tenk2_unique2 on tenk2 t2 (cost=0.29..7.90 rows=1 width=244) (actual time=0.003..0.003 rows=1 loops=10)
Index Cond: (unique2 = t1.unique2)
+ Buffers: shared hit=24 read=6
+ Planning:
+ Buffers: shared hit=15
Planning Time: 0.485 ms
Execution Time: 0.073 ms
</screen>
@@ -769,16 +775,24 @@ WHERE t1.unique1 < 100 AND t1.unique2 = t2.unique2 ORDER BY t1.fivethous;
Sort (cost=713.05..713.30 rows=100 width=488) (actual time=2.995..3.002 rows=100 loops=1)
Sort Key: t1.fivethous
Sort Method: quicksort Memory: 74kB
+ Buffers: shared hit=440
-> Hash Join (cost=226.23..709.73 rows=100 width=488) (actual time=0.515..2.920 rows=100 loops=1)
Hash Cond: (t2.unique2 = t1.unique2)
+ Buffers: shared hit=437
-> Seq Scan on tenk2 t2 (cost=0.00..445.00 rows=10000 width=244) (actual time=0.026..1.790 rows=10000 loops=1)
+ Buffers: shared hit=345
-> Hash (cost=224.98..224.98 rows=100 width=244) (actual time=0.476..0.477 rows=100 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 35kB
+ Buffers: shared hit=92
-> Bitmap Heap Scan on tenk1 t1 (cost=5.06..224.98 rows=100 width=244) (actual time=0.030..0.450 rows=100 loops=1)
Recheck Cond: (unique1 < 100)
Heap Blocks: exact=90
+ Buffers: shared hit=92
-> Bitmap Index Scan on tenk1_unique1 (cost=0.00..5.04 rows=100 width=0) (actual time=0.013..0.013 rows=100 loops=1)
Index Cond: (unique1 < 100)
+ Buffers: shared hit=2
+ Planning:
+ Buffers: shared hit=12
Planning Time: 0.187 ms
Execution Time: 3.036 ms
</screen>
@@ -803,6 +817,7 @@ EXPLAIN ANALYZE SELECT * FROM tenk1 WHERE ten < 7;
Seq Scan on tenk1 (cost=0.00..470.00 rows=7000 width=244) (actual time=0.030..1.995 rows=7000 loops=1)
Filter: (ten < 7)
Rows Removed by Filter: 3000
+ Buffers: shared hit=345
Planning Time: 0.102 ms
Execution Time: 2.145 ms
</screen>
@@ -826,6 +841,7 @@ EXPLAIN ANALYZE SELECT * FROM polygon_tbl WHERE f1 @> polygon '(0.5,2.0)';
Seq Scan on polygon_tbl (cost=0.00..1.09 rows=1 width=85) (actual time=0.023..0.023 rows=0 loops=1)
Filter: (f1 @> '((0.5,2))'::polygon)
Rows Removed by Filter: 7
+ Buffers: shared hit=1
Planning Time: 0.039 ms
Execution Time: 0.033 ms
</screen>
@@ -845,6 +861,7 @@ EXPLAIN ANALYZE SELECT * FROM polygon_tbl WHERE f1 @> polygon '(0.5,2.0)';
Index Scan using gpolygonind on polygon_tbl (cost=0.13..8.15 rows=1 width=85) (actual time=0.074..0.074 rows=0 loops=1)
Index Cond: (f1 @> '((0.5,2))'::polygon)
Rows Removed by Index Recheck: 1
+ Buffers: shared hit=3
Planning Time: 0.039 ms
Execution Time: 0.098 ms
</screen>
@@ -857,34 +874,27 @@ EXPLAIN ANALYZE SELECT * FROM polygon_tbl WHERE f1 @> polygon '(0.5,2.0)';
</para>
<para>
- <command>EXPLAIN</command> has a <literal>BUFFERS</literal> option that can be used with
- <literal>ANALYZE</literal> to get even more run time statistics:
+ <command>EXPLAIN</command> has a <literal>BUFFERS</literal> option which is
+ enabled by default when <literal>ANALYZE</literal> is used. The numbers provided
+ by <literal>BUFFERS</literal> help to identify which parts of the query are the
+ most I/O-intensive. You can turn it off:
<screen>
-EXPLAIN (ANALYZE, BUFFERS) SELECT * FROM tenk1 WHERE unique1 < 100 AND unique2 > 9000;
+EXPLAIN (ANALYZE, BUFFERS OFF) SELECT * FROM tenk1 WHERE unique1 < 100 AND unique2 > 9000;
QUERY PLAN
-------------------------------------------------------------------&zwsp;--------------------------------------------------------------
Bitmap Heap Scan on tenk1 (cost=25.07..60.11 rows=10 width=244) (actual time=0.105..0.114 rows=10 loops=1)
Recheck Cond: ((unique1 < 100) AND (unique2 > 9000))
Heap Blocks: exact=10
- Buffers: shared hit=14 read=3
-> BitmapAnd (cost=25.07..25.07 rows=10 width=0) (actual time=0.100..0.101 rows=0 loops=1)
- Buffers: shared hit=4 read=3
-> Bitmap Index Scan on tenk1_unique1 (cost=0.00..5.04 rows=100 width=0) (actual time=0.027..0.027 rows=100 loops=1)
Index Cond: (unique1 < 100)
- Buffers: shared hit=2
-> Bitmap Index Scan on tenk1_unique2 (cost=0.00..19.78 rows=999 width=0) (actual time=0.070..0.070 rows=999 loops=1)
Index Cond: (unique2 > 9000)
- Buffers: shared hit=2 read=3
- Planning:
- Buffers: shared hit=3
Planning Time: 0.162 ms
Execution Time: 0.143 ms
</screen>
-
- The numbers provided by <literal>BUFFERS</literal> help to identify which parts
- of the query are the most I/O-intensive.
</para>
<para>
@@ -906,8 +916,12 @@ EXPLAIN ANALYZE UPDATE tenk1 SET hundred = hundred + 1 WHERE unique1 < 100;
-> Bitmap Heap Scan on tenk1 (cost=5.06..225.23 rows=100 width=10) (actual time=0.065..0.141 rows=100 loops=1)
Recheck Cond: (unique1 < 100)
Heap Blocks: exact=90
+ Buffers: shared hit=92
-> Bitmap Index Scan on tenk1_unique1 (cost=0.00..5.04 rows=100 width=0) (actual time=0.031..0.031 rows=100 loops=1)
Index Cond: (unique1 < 100)
+ Buffers: shared hit=2
+ Planning:
+ Buffers: shared hit=91
Planning Time: 0.151 ms
Execution Time: 1.856 ms
@@ -1040,10 +1054,14 @@ EXPLAIN ANALYZE SELECT * FROM tenk1 WHERE unique1 < 100 AND unique2 > 9000
QUERY PLAN
-------------------------------------------------------------------&zwsp;------------------------------------------------------------
Limit (cost=0.29..14.33 rows=2 width=244) (actual time=0.051..0.071 rows=2 loops=1)
+ Buffers: shared hit=16
-> Index Scan using tenk1_unique2 on tenk1 (cost=0.29..70.50 rows=10 width=244) (actual time=0.051..0.070 rows=2 loops=1)
Index Cond: (unique2 > 9000)
Filter: (unique1 < 100)
Rows Removed by Filter: 287
+ Buffers: shared hit=16
+ Planning:
+ Buffers: shared hit=49
Planning Time: 0.077 ms
Execution Time: 0.086 ms
</screen>
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index db9d3a8549..4d6f655b03 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -201,8 +201,10 @@ ROLLBACK;
query processing.
The number of blocks shown for an
upper-level node includes those used by all its child nodes. In text
- format, only non-zero values are printed. This parameter defaults to
- <literal>FALSE</literal>.
+ format, only non-zero values are printed.
+ Buffers information is included by default when
+ <literal>ANALYZE</literal> is used but otherwise is not included by
+ default, but can be enabled using this option.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 7c0fd63b2f..d6815e01d1 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -198,6 +198,7 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
List *rewritten;
ListCell *lc;
bool timing_set = false;
+ bool buffers_set = false;
bool summary_set = false;
/* Parse options list. */
@@ -212,7 +213,10 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
else if (strcmp(opt->defname, "costs") == 0)
es->costs = defGetBoolean(opt);
else if (strcmp(opt->defname, "buffers") == 0)
+ {
+ buffers_set = true;
es->buffers = defGetBoolean(opt);
+ }
else if (strcmp(opt->defname, "wal") == 0)
es->wal = defGetBoolean(opt);
else if (strcmp(opt->defname, "settings") == 0)
@@ -292,6 +296,9 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
/* if the timing was not set explicitly, set default value */
es->timing = (timing_set) ? es->timing : es->analyze;
+ /* if the buffers was not set explicitly, set default value */
+ es->buffers = (buffers_set) ? es->buffers : es->analyze;
+
/* check that timing is used with EXPLAIN ANALYZE */
if (es->timing && !es->analyze)
ereport(ERROR,
diff --git a/src/test/regress/expected/brin_multi.out b/src/test/regress/expected/brin_multi.out
index ae9ce9d8ec..f2d1465818 100644
--- a/src/test/regress/expected/brin_multi.out
+++ b/src/test/regress/expected/brin_multi.out
@@ -845,7 +845,7 @@ INSERT INTO brin_date_test SELECT '5874897-12-01'::date + i FROM generate_series
CREATE INDEX ON brin_date_test USING brin (a date_minmax_multi_ops) WITH (pages_per_range=1);
SET enable_seqscan = off;
-- make sure the ranges were built correctly and 2023-01-01 eliminates all
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_date_test WHERE a = '2023-01-01'::date;
QUERY PLAN
-------------------------------------------------------------------------
@@ -864,7 +864,7 @@ INSERT INTO brin_timestamp_test
SELECT i FROM generate_series('2000-01-01'::timestamp, '2000-02-09'::timestamp, '1 day'::interval) s(i);
CREATE INDEX ON brin_timestamp_test USING brin (a timestamp_minmax_multi_ops) WITH (pages_per_range=1);
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_timestamp_test WHERE a = '2023-01-01'::timestamp;
QUERY PLAN
------------------------------------------------------------------------------
@@ -874,7 +874,7 @@ SELECT * FROM brin_timestamp_test WHERE a = '2023-01-01'::timestamp;
Index Cond: (a = '2023-01-01 00:00:00'::timestamp without time zone)
(4 rows)
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_timestamp_test WHERE a = '1900-01-01'::timestamp;
QUERY PLAN
------------------------------------------------------------------------------
@@ -892,7 +892,7 @@ INSERT INTO brin_date_test VALUES ('-infinity'), ('infinity');
INSERT INTO brin_date_test SELECT '2000-01-01'::date + i FROM generate_series(1, 40) s(i);
CREATE INDEX ON brin_date_test USING brin (a date_minmax_multi_ops) WITH (pages_per_range=1);
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_date_test WHERE a = '2023-01-01'::date;
QUERY PLAN
-------------------------------------------------------------------------
@@ -902,7 +902,7 @@ SELECT * FROM brin_date_test WHERE a = '2023-01-01'::date;
Index Cond: (a = '2023-01-01'::date)
(4 rows)
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_date_test WHERE a = '1900-01-01'::date;
QUERY PLAN
-------------------------------------------------------------------------
@@ -921,7 +921,7 @@ INSERT INTO brin_interval_test SELECT (i || ' years')::interval FROM generate_se
INSERT INTO brin_interval_test SELECT (i || ' years')::interval FROM generate_series( 177999980, 178000000) s(i);
CREATE INDEX ON brin_interval_test USING brin (a interval_minmax_multi_ops) WITH (pages_per_range=1);
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '-30 years'::interval;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -931,7 +931,7 @@ SELECT * FROM brin_interval_test WHERE a = '-30 years'::interval;
Index Cond: (a = '@ 30 years ago'::interval)
(4 rows)
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '30 years'::interval;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -949,7 +949,7 @@ INSERT INTO brin_interval_test VALUES ('-infinity'), ('infinity');
INSERT INTO brin_interval_test SELECT (i || ' days')::interval FROM generate_series(100, 140) s(i);
CREATE INDEX ON brin_interval_test USING brin (a interval_minmax_multi_ops) WITH (pages_per_range=1);
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '-30 years'::interval;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -959,7 +959,7 @@ SELECT * FROM brin_interval_test WHERE a = '-30 years'::interval;
Index Cond: (a = '@ 30 years ago'::interval)
(4 rows)
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '30 years'::interval;
QUERY PLAN
-----------------------------------------------------------------------------
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index d2eef8097c..930b4c6f48 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -77,7 +77,7 @@ select explain_filter('explain (analyze, verbose) select * from int8_tbl i8');
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (analyze, format text) select * from int8_tbl i8');
explain_filter
-----------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N loops=N)
@@ -85,7 +85,7 @@ select explain_filter('explain (analyze, buffers, format text) select * from int
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, format xml) select * from int8_tbl i8');
explain_filter
--------------------------------------------------------
<explain xmlns="http://www.postgresql.org/N/explain"> +
@@ -136,7 +136,15 @@ select explain_filter('explain (analyze, buffers, format xml) select * from int8
</explain>
(1 row)
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, format text) select * from int8_tbl i8');
+ explain_filter
+-----------------------------------------------------------------------------------------------
+ Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N loops=N)
+ Planning Time: N.N ms
+ Execution Time: N.N ms
+(3 rows)
+
+select explain_filter('explain (analyze, serialize, format yaml) select * from int8_tbl i8');
explain_filter
-------------------------------
- Plan: +
@@ -400,9 +408,29 @@ select explain_filter('explain (memory, analyze, format json) select * from int8
"Actual Total Time": N.N, +
"Actual Rows": N, +
"Actual Loops": N, +
- "Disabled": false +
+ "Disabled": false, +
+ "Shared Hit Blocks": N, +
+ "Shared Read Blocks": N, +
+ "Shared Dirtied Blocks": N, +
+ "Shared Written Blocks": N, +
+ "Local Hit Blocks": N, +
+ "Local Read Blocks": N, +
+ "Local Dirtied Blocks": N, +
+ "Local Written Blocks": N, +
+ "Temp Read Blocks": N, +
+ "Temp Written Blocks": N +
}, +
"Planning": { +
+ "Shared Hit Blocks": N, +
+ "Shared Read Blocks": N, +
+ "Shared Dirtied Blocks": N, +
+ "Shared Written Blocks": N, +
+ "Local Hit Blocks": N, +
+ "Local Read Blocks": N, +
+ "Local Dirtied Blocks": N, +
+ "Local Written Blocks": N, +
+ "Temp Read Blocks": N, +
+ "Temp Written Blocks": N, +
"Memory Used": N, +
"Memory Allocated": N +
}, +
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 2df7a5db12..d597575840 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -39,7 +39,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -55,7 +55,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/expected/memoize.out b/src/test/regress/expected/memoize.out
index f6b8329cd6..5ecf971dad 100644
--- a/src/test/regress/expected/memoize.out
+++ b/src/test/regress/expected/memoize.out
@@ -10,7 +10,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/expected/merge.out b/src/test/regress/expected/merge.out
index 521d70a891..28d8551063 100644
--- a/src/test/regress/expected/merge.out
+++ b/src/test/regress/expected/merge.out
@@ -1621,7 +1621,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 7a03b4e360..c710f316d7 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
@@ -2127,7 +2127,7 @@ create table ab_a3_b3 partition of ab_a3 for values in (3);
set enable_indexonlyscan = off;
prepare ab_q1 (int, int, int) as
select * from ab where a between $1 and $2 and b <= $3;
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2, 3);
QUERY PLAN
---------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2140,7 +2140,7 @@ explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 2, 3);
Filter: ((a >= $1) AND (a <= $2) AND (b <= $3))
(8 rows)
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (1, 2, 3);
QUERY PLAN
---------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2163,7 +2163,7 @@ deallocate ab_q1;
-- Runtime pruning after optimizer pruning
prepare ab_q1 (int, int) as
select a from ab where a between $1 and $2 and b < 3;
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2);
QUERY PLAN
---------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2174,7 +2174,7 @@ explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 2);
Filter: ((a >= $1) AND (a <= $2) AND (b < 3))
(6 rows)
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 4);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 4);
QUERY PLAN
---------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2193,7 +2193,7 @@ explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 4);
-- different levels of partitioning.
prepare ab_q2 (int, int) as
select a from ab where a between $1 and $2 and b < (select 3);
-explain (analyze, costs off, summary off, timing off) execute ab_q2 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q2 (2, 2);
QUERY PLAN
-----------------------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2211,7 +2211,7 @@ explain (analyze, costs off, summary off, timing off) execute ab_q2 (2, 2);
-- As above, but swap the PARAM_EXEC Param to the first partition level
prepare ab_q3 (int, int) as
select a from ab where b between $1 and $2 and a < (select 3);
-explain (analyze, costs off, summary off, timing off) execute ab_q3 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q3 (2, 2);
QUERY PLAN
-----------------------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2273,7 +2273,7 @@ begin;
-- Test run-time pruning using stable functions
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
QUERY PLAN
------------------------------------------------------------------
Append (actual rows=1 loops=1)
@@ -2283,7 +2283,7 @@ explain (analyze, costs off, summary off, timing off) select * from list_part wh
(4 rows)
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
QUERY PLAN
------------------------------------------------------------------
Append (actual rows=4 loops=1)
@@ -2298,7 +2298,7 @@ explain (analyze, costs off, summary off, timing off) select * from list_part wh
(9 rows)
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
QUERY PLAN
------------------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2334,7 +2334,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -2641,7 +2641,7 @@ reset parallel_tuple_cost;
reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
QUERY PLAN
-------------------------------------------------------------------------
@@ -2700,7 +2700,7 @@ select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1
(52 rows)
-- Test run-time partition pruning with UNION ALL parents
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from (select * from ab where a = 1 union all select * from ab) ab where b = (select 1);
QUERY PLAN
-------------------------------------------------------------------------------
@@ -2744,7 +2744,7 @@ select * from (select * from ab where a = 1 union all select * from ab) ab where
(37 rows)
-- A case containing a UNION ALL with a non-partitioned child.
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from (select * from ab where a = 1 union all (values(10,5)) union all select * from ab) ab where b = (select 1);
QUERY PLAN
-------------------------------------------------------------------------------
@@ -2803,7 +2803,7 @@ union all
select tableoid::regclass,a,b from ab
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
QUERY PLAN
--------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2952,7 +2952,7 @@ create index tprt6_idx on tprt_6 (col1);
insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
--------------------------------------------------------------------------
@@ -2973,7 +2973,7 @@ select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
Index Cond: (col1 < tbl1.col1)
(15 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
--------------------------------------------------------------------------
@@ -3018,7 +3018,7 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
--------------------------------------------------------------------------
@@ -3039,7 +3039,7 @@ select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
Index Cond: (col1 < tbl1.col1)
(15 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
--------------------------------------------------------------------------
@@ -3103,7 +3103,7 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
QUERY PLAN
--------------------------------------------------------------------------
@@ -3135,7 +3135,7 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-------------------------------------------------------------------
@@ -3175,7 +3175,7 @@ alter table part_cab attach partition part_abc_p1 for values in(3);
prepare part_abc_q1 (int, int, int) as
select * from part_abc where a = $1 and b = $2 and c = $3;
-- Single partition should be scanned.
-explain (analyze, costs off, summary off, timing off) execute part_abc_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off) execute part_abc_q1 (1, 2, 3);
QUERY PLAN
----------------------------------------------------------
Seq Scan on part_abc_p1 part_abc (actual rows=0 loops=1)
@@ -3200,7 +3200,7 @@ select * from listp where b = 1;
-- partitions before finally detecting the correct set of 2nd level partitions
-- which match the given parameter.
prepare q1 (int,int) as select * from listp where b in ($1,$2);
-explain (analyze, costs off, summary off, timing off) execute q1 (1,1);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,1);
QUERY PLAN
-------------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -3209,7 +3209,7 @@ explain (analyze, costs off, summary off, timing off) execute q1 (1,1);
Filter: (b = ANY (ARRAY[$1, $2]))
(4 rows)
-explain (analyze, costs off, summary off, timing off) execute q1 (2,2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (2,2);
QUERY PLAN
-------------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -3219,7 +3219,7 @@ explain (analyze, costs off, summary off, timing off) execute q1 (2,2);
(4 rows)
-- Try with no matching partitions.
-explain (analyze, costs off, summary off, timing off) execute q1 (0,0);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (0,0);
QUERY PLAN
--------------------------------
Append (actual rows=0 loops=1)
@@ -3230,7 +3230,7 @@ deallocate q1;
-- Test more complex cases where a not-equal condition further eliminates partitions.
prepare q1 (int,int,int,int) as select * from listp where b in($1,$2) and $3 <> b and $4 <> b;
-- Both partitions allowed by IN clause, but one disallowed by <> clause
-explain (analyze, costs off, summary off, timing off) execute q1 (1,2,2,0);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,0);
QUERY PLAN
-------------------------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -3240,7 +3240,7 @@ explain (analyze, costs off, summary off, timing off) execute q1 (1,2,2,0);
(4 rows)
-- Both partitions allowed by IN clause, then both excluded again by <> clauses.
-explain (analyze, costs off, summary off, timing off) execute q1 (1,2,2,1);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,1);
QUERY PLAN
--------------------------------
Append (actual rows=0 loops=1)
@@ -3248,7 +3248,7 @@ explain (analyze, costs off, summary off, timing off) execute q1 (1,2,2,1);
(2 rows)
-- Ensure Params that evaluate to NULL properly prune away all partitions
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from listp where a = (select null::int);
QUERY PLAN
------------------------------------------------------
@@ -3273,7 +3273,7 @@ create table stable_qual_pruning2 partition of stable_qual_pruning
create table stable_qual_pruning3 partition of stable_qual_pruning
for values from ('3000-02-01') to ('3000-03-01');
-- comparison against a stable value requires run-time pruning
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning where a < localtimestamp;
QUERY PLAN
--------------------------------------------------------------------------------------
@@ -3286,7 +3286,7 @@ select * from stable_qual_pruning where a < localtimestamp;
(6 rows)
-- timestamp < timestamptz comparison is only stable, not immutable
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning where a < '2000-02-01'::timestamptz;
QUERY PLAN
--------------------------------------------------------------------------------------
@@ -3297,7 +3297,7 @@ select * from stable_qual_pruning where a < '2000-02-01'::timestamptz;
(4 rows)
-- check ScalarArrayOp cases
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamp[]);
QUERY PLAN
@@ -3306,7 +3306,7 @@ select * from stable_qual_pruning
One-Time Filter: false
(2 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamp[]);
QUERY PLAN
@@ -3315,7 +3315,7 @@ select * from stable_qual_pruning
Filter: (a = ANY ('{"Tue Feb 01 00:00:00 2000","Fri Jan 01 00:00:00 2010"}'::timestamp without time zone[]))
(2 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', localtimestamp]::timestamp[]);
QUERY PLAN
@@ -3326,7 +3326,7 @@ select * from stable_qual_pruning
Filter: (a = ANY (ARRAY['Tue Feb 01 00:00:00 2000'::timestamp without time zone, LOCALTIMESTAMP]))
(4 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamptz[]);
QUERY PLAN
@@ -3335,7 +3335,7 @@ select * from stable_qual_pruning
Subplans Removed: 3
(2 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamptz[]);
QUERY PLAN
@@ -3346,7 +3346,7 @@ select * from stable_qual_pruning
Filter: (a = ANY ('{"Tue Feb 01 00:00:00 2000 PST","Fri Jan 01 00:00:00 2010 PST"}'::timestamp with time zone[]))
(4 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(null::timestamptz[]);
QUERY PLAN
@@ -3374,7 +3374,7 @@ create table mc3p1 partition of mc3p
create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from mc3p where a < 3 and abs(b) = 1;
QUERY PLAN
--------------------------------------------------------
@@ -3394,7 +3394,7 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
execute ps1(1);
QUERY PLAN
-------------------------------------------------------------
@@ -3409,7 +3409,7 @@ execute ps1(1);
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
execute ps2(1);
QUERY PLAN
--------------------------------------------------------------
@@ -3431,7 +3431,7 @@ insert into boolvalues values('t'),('f');
create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from boolp where a = (select value from boolvalues where value);
QUERY PLAN
-----------------------------------------------------------
@@ -3446,7 +3446,7 @@ select * from boolp where a = (select value from boolvalues where value);
Filter: (a = (InitPlan 1).col1)
(9 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from boolp where a = (select value from boolvalues where not value);
QUERY PLAN
-----------------------------------------------------------
@@ -3475,7 +3475,7 @@ insert into ma_test select x,x from generate_series(0,29) t(x);
create index on ma_test (b);
analyze ma_test;
prepare mt_q1 (int) as select a from ma_test where a >= $1 and a % 10 = 5 order by b;
-explain (analyze, costs off, summary off, timing off) execute mt_q1(15);
+explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(15);
QUERY PLAN
-----------------------------------------------------------------------------------------
Merge Append (actual rows=2 loops=1)
@@ -3496,7 +3496,7 @@ execute mt_q1(15);
25
(2 rows)
-explain (analyze, costs off, summary off, timing off) execute mt_q1(25);
+explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(25);
QUERY PLAN
-----------------------------------------------------------------------------------------
Merge Append (actual rows=1 loops=1)
@@ -3514,7 +3514,7 @@ execute mt_q1(25);
(1 row)
-- Ensure MergeAppend behaves correctly when no subplans match
-explain (analyze, costs off, summary off, timing off) execute mt_q1(35);
+explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(35);
QUERY PLAN
--------------------------------------
Merge Append (actual rows=0 loops=1)
@@ -3542,7 +3542,7 @@ explain (analyze, verbose, costs off, summary off, timing off) execute mt_q2 (35
deallocate mt_q2;
-- ensure initplan params properly prune partitions
-explain (analyze, costs off, summary off, timing off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
+explain (analyze, costs off, summary off, timing off, buffers off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
QUERY PLAN
-----------------------------------------------------------------------------------------------
Merge Append (actual rows=20 loops=1)
@@ -3992,7 +3992,7 @@ create table listp (a int, b int) partition by list (a);
create table listp1 partition of listp for values in(1);
create table listp2 partition of listp for values in(2) partition by list(b);
create table listp2_10 partition of listp2 for values in (10);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from listp where a = (select 2) and b <> 10;
QUERY PLAN
---------------------------------------------------
@@ -4117,7 +4117,7 @@ create table rangep_0_to_100_3 partition of rangep_0_to_100 for values in(3);
create table rangep_100_to_200 partition of rangep for values from (100) to (200);
create index on rangep (a);
-- Ensure run-time pruning works on the nested Merge Append
-explain (analyze on, costs off, timing off, summary off)
+explain (analyze on, costs off, timing off, summary off, buffers off)
select * from rangep where b IN((select 1),(select 2)) order by a;
QUERY PLAN
------------------------------------------------------------------------------------------------------------
diff --git a/src/test/regress/expected/select.out b/src/test/regress/expected/select.out
index 33a6dceb0e..88911ca2b9 100644
--- a/src/test/regress/expected/select.out
+++ b/src/test/regress/expected/select.out
@@ -757,7 +757,7 @@ select * from onek2 where unique2 = 11 and stringu1 = 'ATAAAA';
(1 row)
-- actually run the query with an analyze to use the partial index
-explain (costs off, analyze on, timing off, summary off)
+explain (costs off, analyze on, timing off, summary off, buffers off)
select * from onek2 where unique2 = 11 and stringu1 = 'ATAAAA';
QUERY PLAN
-----------------------------------------------------------------
diff --git a/src/test/regress/expected/select_into.out b/src/test/regress/expected/select_into.out
index b79fe9a1c0..d6578d992c 100644
--- a/src/test/regress/expected/select_into.out
+++ b/src/test/regress/expected/select_into.out
@@ -25,7 +25,7 @@ CREATE TABLE selinto_schema.tbl_withdata1 (a)
AS SELECT generate_series(1,3) WITH DATA;
INSERT INTO selinto_schema.tbl_withdata1 VALUES (4);
ERROR: permission denied for table tbl_withdata1
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
CREATE TABLE selinto_schema.tbl_withdata2 (a) AS
SELECT generate_series(1,3) WITH DATA;
QUERY PLAN
@@ -62,7 +62,7 @@ EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
-- EXECUTE and WITH NO DATA, passes.
CREATE TABLE selinto_schema.tbl_nodata3 (a) AS
EXECUTE data_sel WITH NO DATA;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
CREATE TABLE selinto_schema.tbl_nodata4 (a) AS
EXECUTE data_sel WITH NO DATA;
QUERY PLAN
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 8c31f6460d..a809036453 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -580,7 +580,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off)
+explain (analyze, timing off, summary off, costs off, buffers off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
QUERY PLAN
@@ -606,7 +606,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off)
+ explain (analyze, timing off, summary off, costs off, buffers off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -1169,7 +1169,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
QUERY PLAN
-------------------------------------------------------------
Gather (actual rows=10000 loops=1)
diff --git a/src/test/regress/expected/subselect.out b/src/test/regress/expected/subselect.out
index 2d35de3fad..e04cea3d36 100644
--- a/src/test/regress/expected/subselect.out
+++ b/src/test/regress/expected/subselect.out
@@ -1675,7 +1675,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off)
+ explain (analyze, summary off, timing off, costs off, buffers off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
diff --git a/src/test/regress/expected/tidscan.out b/src/test/regress/expected/tidscan.out
index f133b5a4ac..f6ebdf0601 100644
--- a/src/test/regress/expected/tidscan.out
+++ b/src/test/regress/expected/tidscan.out
@@ -189,7 +189,7 @@ FETCH NEXT FROM c;
(1 row)
-- perform update
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
UPDATE tidscan SET id = -id WHERE CURRENT OF c RETURNING *;
QUERY PLAN
---------------------------------------------------
@@ -205,7 +205,7 @@ FETCH NEXT FROM c;
(1 row)
-- perform update
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
UPDATE tidscan SET id = -id WHERE CURRENT OF c RETURNING *;
QUERY PLAN
---------------------------------------------------
@@ -229,7 +229,7 @@ FETCH NEXT FROM c;
(0 rows)
-- should error out
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
UPDATE tidscan SET id = -id WHERE CURRENT OF c RETURNING *;
ERROR: cursor "c" is not positioned on a row
ROLLBACK;
diff --git a/src/test/regress/sql/brin_multi.sql b/src/test/regress/sql/brin_multi.sql
index 55349b4e1f..dda9fb73e0 100644
--- a/src/test/regress/sql/brin_multi.sql
+++ b/src/test/regress/sql/brin_multi.sql
@@ -619,7 +619,7 @@ CREATE INDEX ON brin_date_test USING brin (a date_minmax_multi_ops) WITH (pages_
SET enable_seqscan = off;
-- make sure the ranges were built correctly and 2023-01-01 eliminates all
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_date_test WHERE a = '2023-01-01'::date;
DROP TABLE brin_date_test;
@@ -636,10 +636,10 @@ CREATE INDEX ON brin_timestamp_test USING brin (a timestamp_minmax_multi_ops) WI
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_timestamp_test WHERE a = '2023-01-01'::timestamp;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_timestamp_test WHERE a = '1900-01-01'::timestamp;
DROP TABLE brin_timestamp_test;
@@ -655,10 +655,10 @@ CREATE INDEX ON brin_date_test USING brin (a date_minmax_multi_ops) WITH (pages_
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_date_test WHERE a = '2023-01-01'::date;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_date_test WHERE a = '1900-01-01'::date;
DROP TABLE brin_date_test;
@@ -676,10 +676,10 @@ CREATE INDEX ON brin_interval_test USING brin (a interval_minmax_multi_ops) WITH
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '-30 years'::interval;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '30 years'::interval;
DROP TABLE brin_interval_test;
@@ -695,10 +695,10 @@ CREATE INDEX ON brin_interval_test USING brin (a interval_minmax_multi_ops) WITH
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '-30 years'::interval;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '30 years'::interval;
DROP TABLE brin_interval_test;
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index 3ca285a1d7..3f41396886 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -64,9 +64,10 @@ set track_io_timing = off;
select explain_filter('explain select * from int8_tbl i8');
select explain_filter('explain (analyze) select * from int8_tbl i8');
select explain_filter('explain (analyze, verbose) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, format text) select * from int8_tbl i8');
+select explain_filter('explain (analyze, format xml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, format text) select * from int8_tbl i8');
+select explain_filter('explain (analyze, serialize, format yaml) select * from int8_tbl i8');
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
diff --git a/src/test/regress/sql/incremental_sort.sql b/src/test/regress/sql/incremental_sort.sql
index 98b20e17e1..f1f8fae565 100644
--- a/src/test/regress/sql/incremental_sort.sql
+++ b/src/test/regress/sql/incremental_sort.sql
@@ -21,7 +21,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -38,7 +38,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/sql/memoize.sql b/src/test/regress/sql/memoize.sql
index 2eaeb1477a..d5aab4e566 100644
--- a/src/test/regress/sql/memoize.sql
+++ b/src/test/regress/sql/memoize.sql
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/sql/merge.sql b/src/test/regress/sql/merge.sql
index 5ddcca84f8..54929a92fa 100644
--- a/src/test/regress/sql/merge.sql
+++ b/src/test/regress/sql/merge.sql
@@ -1072,7 +1072,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index 442428d937..8cd4a21fdc 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -12,7 +12,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
@@ -465,8 +465,8 @@ set enable_indexonlyscan = off;
prepare ab_q1 (int, int, int) as
select * from ab where a between $1 and $2 and b <= $3;
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 2, 3);
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (1, 2, 3);
deallocate ab_q1;
@@ -474,21 +474,21 @@ deallocate ab_q1;
prepare ab_q1 (int, int) as
select a from ab where a between $1 and $2 and b < 3;
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 2);
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 4);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 4);
-- Ensure a mix of PARAM_EXTERN and PARAM_EXEC Params work together at
-- different levels of partitioning.
prepare ab_q2 (int, int) as
select a from ab where a between $1 and $2 and b < (select 3);
-explain (analyze, costs off, summary off, timing off) execute ab_q2 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q2 (2, 2);
-- As above, but swap the PARAM_EXEC Param to the first partition level
prepare ab_q3 (int, int) as
select a from ab where b between $1 and $2 and a < (select 3);
-explain (analyze, costs off, summary off, timing off) execute ab_q3 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q3 (2, 2);
--
-- Test runtime pruning with hash partitioned tables
@@ -538,13 +538,13 @@ begin;
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
rollback;
@@ -567,7 +567,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -650,15 +650,15 @@ reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
-- Test run-time partition pruning with UNION ALL parents
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from (select * from ab where a = 1 union all select * from ab) ab where b = (select 1);
-- A case containing a UNION ALL with a non-partitioned child.
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from (select * from ab where a = 1 union all (values(10,5)) union all select * from ab) ab where b = (select 1);
-- Another UNION ALL test, but containing a mix of exec init and exec run-time pruning.
@@ -678,7 +678,7 @@ union all
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
-- Ensure we see just the xy_1 row.
execute ab_q6(100);
@@ -733,10 +733,10 @@ insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -749,10 +749,10 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -766,7 +766,7 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -776,7 +776,7 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -799,7 +799,7 @@ prepare part_abc_q1 (int, int, int) as
select * from part_abc where a = $1 and b = $2 and c = $3;
-- Single partition should be scanned.
-explain (analyze, costs off, summary off, timing off) execute part_abc_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off) execute part_abc_q1 (1, 2, 3);
deallocate part_abc_q1;
@@ -819,12 +819,12 @@ select * from listp where b = 1;
-- which match the given parameter.
prepare q1 (int,int) as select * from listp where b in ($1,$2);
-explain (analyze, costs off, summary off, timing off) execute q1 (1,1);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,1);
-explain (analyze, costs off, summary off, timing off) execute q1 (2,2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (2,2);
-- Try with no matching partitions.
-explain (analyze, costs off, summary off, timing off) execute q1 (0,0);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (0,0);
deallocate q1;
@@ -832,13 +832,13 @@ deallocate q1;
prepare q1 (int,int,int,int) as select * from listp where b in($1,$2) and $3 <> b and $4 <> b;
-- Both partitions allowed by IN clause, but one disallowed by <> clause
-explain (analyze, costs off, summary off, timing off) execute q1 (1,2,2,0);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,0);
-- Both partitions allowed by IN clause, then both excluded again by <> clauses.
-explain (analyze, costs off, summary off, timing off) execute q1 (1,2,2,1);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,1);
-- Ensure Params that evaluate to NULL properly prune away all partitions
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from listp where a = (select null::int);
drop table listp;
@@ -855,30 +855,30 @@ create table stable_qual_pruning3 partition of stable_qual_pruning
for values from ('3000-02-01') to ('3000-03-01');
-- comparison against a stable value requires run-time pruning
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning where a < localtimestamp;
-- timestamp < timestamptz comparison is only stable, not immutable
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning where a < '2000-02-01'::timestamptz;
-- check ScalarArrayOp cases
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamp[]);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamp[]);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', localtimestamp]::timestamp[]);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamptz[]);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamptz[]);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(null::timestamptz[]);
@@ -898,7 +898,7 @@ create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from mc3p where a < 3 and abs(b) = 1;
--
@@ -908,12 +908,12 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
execute ps1(1);
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
execute ps2(1);
deallocate ps2;
@@ -927,10 +927,10 @@ create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from boolp where a = (select value from boolvalues where value);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from boolp where a = (select value from boolvalues where not value);
drop table boolp;
@@ -950,12 +950,12 @@ create index on ma_test (b);
analyze ma_test;
prepare mt_q1 (int) as select a from ma_test where a >= $1 and a % 10 = 5 order by b;
-explain (analyze, costs off, summary off, timing off) execute mt_q1(15);
+explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(15);
execute mt_q1(15);
-explain (analyze, costs off, summary off, timing off) execute mt_q1(25);
+explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(25);
execute mt_q1(25);
-- Ensure MergeAppend behaves correctly when no subplans match
-explain (analyze, costs off, summary off, timing off) execute mt_q1(35);
+explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(35);
execute mt_q1(35);
deallocate mt_q1;
@@ -968,7 +968,7 @@ explain (analyze, verbose, costs off, summary off, timing off) execute mt_q2 (35
deallocate mt_q2;
-- ensure initplan params properly prune partitions
-explain (analyze, costs off, summary off, timing off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
+explain (analyze, costs off, summary off, timing off, buffers off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
reset enable_seqscan;
reset enable_sort;
@@ -1148,7 +1148,7 @@ create table listp1 partition of listp for values in(1);
create table listp2 partition of listp for values in(2) partition by list(b);
create table listp2_10 partition of listp2 for values in (10);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from listp where a = (select 2) and b <> 10;
--
@@ -1216,7 +1216,7 @@ create table rangep_100_to_200 partition of rangep for values from (100) to (200
create index on rangep (a);
-- Ensure run-time pruning works on the nested Merge Append
-explain (analyze on, costs off, timing off, summary off)
+explain (analyze on, costs off, timing off, summary off, buffers off)
select * from rangep where b IN((select 1),(select 2)) order by a;
reset enable_sort;
drop table rangep;
diff --git a/src/test/regress/sql/select.sql b/src/test/regress/sql/select.sql
index 019f1e7673..1d1bf2b931 100644
--- a/src/test/regress/sql/select.sql
+++ b/src/test/regress/sql/select.sql
@@ -196,7 +196,7 @@ explain (costs off)
select * from onek2 where unique2 = 11 and stringu1 = 'ATAAAA';
select * from onek2 where unique2 = 11 and stringu1 = 'ATAAAA';
-- actually run the query with an analyze to use the partial index
-explain (costs off, analyze on, timing off, summary off)
+explain (costs off, analyze on, timing off, summary off, buffers off)
select * from onek2 where unique2 = 11 and stringu1 = 'ATAAAA';
explain (costs off)
select unique2 from onek2 where unique2 = 11 and stringu1 = 'ATAAAA';
diff --git a/src/test/regress/sql/select_into.sql b/src/test/regress/sql/select_into.sql
index 689c448cc2..106cdde187 100644
--- a/src/test/regress/sql/select_into.sql
+++ b/src/test/regress/sql/select_into.sql
@@ -30,7 +30,7 @@ SET SESSION AUTHORIZATION regress_selinto_user;
CREATE TABLE selinto_schema.tbl_withdata1 (a)
AS SELECT generate_series(1,3) WITH DATA;
INSERT INTO selinto_schema.tbl_withdata1 VALUES (4);
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
CREATE TABLE selinto_schema.tbl_withdata2 (a) AS
SELECT generate_series(1,3) WITH DATA;
-- WITH NO DATA, passes.
@@ -49,7 +49,7 @@ EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
-- EXECUTE and WITH NO DATA, passes.
CREATE TABLE selinto_schema.tbl_nodata3 (a) AS
EXECUTE data_sel WITH NO DATA;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
CREATE TABLE selinto_schema.tbl_nodata4 (a) AS
EXECUTE data_sel WITH NO DATA;
RESET SESSION AUTHORIZATION;
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 5b4a6e1088..71a75bc86e 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -230,7 +230,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off)
+explain (analyze, timing off, summary off, costs off, buffers off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
alter table tenk2 reset (parallel_workers);
@@ -242,7 +242,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off)
+ explain (analyze, timing off, summary off, costs off, buffers off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -450,7 +450,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
ROLLBACK TO SAVEPOINT settings;
-- provoke error in worker
diff --git a/src/test/regress/sql/subselect.sql b/src/test/regress/sql/subselect.sql
index af6e157aca..c53c7f724c 100644
--- a/src/test/regress/sql/subselect.sql
+++ b/src/test/regress/sql/subselect.sql
@@ -857,7 +857,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off)
+ explain (analyze, summary off, timing off, costs off, buffers off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
diff --git a/src/test/regress/sql/tidscan.sql b/src/test/regress/sql/tidscan.sql
index 313e0fb9b6..1b82d5f1a5 100644
--- a/src/test/regress/sql/tidscan.sql
+++ b/src/test/regress/sql/tidscan.sql
@@ -68,17 +68,17 @@ DECLARE c CURSOR FOR SELECT ctid, * FROM tidscan;
FETCH NEXT FROM c; -- skip one row
FETCH NEXT FROM c;
-- perform update
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
UPDATE tidscan SET id = -id WHERE CURRENT OF c RETURNING *;
FETCH NEXT FROM c;
-- perform update
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
UPDATE tidscan SET id = -id WHERE CURRENT OF c RETURNING *;
SELECT * FROM tidscan;
-- position cursor past any rows
FETCH NEXT FROM c;
-- should error out
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
UPDATE tidscan SET id = -id WHERE CURRENT OF c RETURNING *;
ROLLBACK;
--
2.47.0
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-11-20 15:50 Greg Sabino Mullane <[email protected]>
parent: Guillaume Lelarge <[email protected]>
0 siblings, 1 reply; 32+ messages in thread
From: Greg Sabino Mullane @ 2024-11-20 15:50 UTC (permalink / raw)
To: Guillaume Lelarge <[email protected]>; +Cc: Robert Haas <[email protected]>; Michael Christofides <[email protected]>; David Rowley <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Wed, Nov 20, 2024 at 1:26 AM Guillaume Lelarge <[email protected]>
wrote:
> OK, but I'm not sure which example I should pick to add dirtied and
> written shared buffers. It looks kinda artificial. Should I choose one
> randomly?
>
It will be artificial, but I think that's ok: anyone running it on their
own will be getting different numbers anyway. I was looking at the "14.1.2
EXPLAIN ANALYZE" section in perform.sgml. Here's some actual numbers I got
with some playing around with concurrent updates:
Recheck Cond: (unique1 < 10)
> Heap Blocks: exact=5
> Buffers: shared hit=2 read=5 written=4
...
> Planning:
> Buffers: shared hit=289 dirtied=9
Cheers,
Greg
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-11-20 19:30 Guillaume Lelarge <[email protected]>
parent: Greg Sabino Mullane <[email protected]>
0 siblings, 2 replies; 32+ messages in thread
From: Guillaume Lelarge @ 2024-11-20 19:30 UTC (permalink / raw)
To: Greg Sabino Mullane <[email protected]>; +Cc: Robert Haas <[email protected]>; Michael Christofides <[email protected]>; David Rowley <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
Le mer. 20 nov. 2024 à 16:51, Greg Sabino Mullane <[email protected]> a
écrit :
> On Wed, Nov 20, 2024 at 1:26 AM Guillaume Lelarge <[email protected]>
> wrote:
>
>> OK, but I'm not sure which example I should pick to add dirtied and
>> written shared buffers. It looks kinda artificial. Should I choose one
>> randomly?
>>
>
> It will be artificial, but I think that's ok: anyone running it on their
> own will be getting different numbers anyway. I was looking at the "14.1.2
> EXPLAIN ANALYZE" section in perform.sgml. Here's some actual numbers I got
> with some playing around with concurrent updates:
>
> Recheck Cond: (unique1 < 10)
>> Heap Blocks: exact=5
>> Buffers: shared hit=2 read=5 written=4
>
> ...
>
>> Planning:
>> Buffers: shared hit=289 dirtied=9
>
>
>
OK, I'm fine with this. v4 patch attached with one plan showing read,
written, and dirtied buffers.
Thanks for all your comments/reviews.
--
Guillaume.
Attachments:
[text/x-patch] v4-0001-Enable-BUFFERS-by-default-with-EXPLAIN-ANALYZE.patch (77.0K, ../../CAECtzeVkkmvmHWVOTzwpX_GX+k4h7EWAyfERqes9C0z4pzMN2A@mail.gmail.com/3-v4-0001-Enable-BUFFERS-by-default-with-EXPLAIN-ANALYZE.patch)
download | inline diff:
From 484779613694f777e2ffb815a16aa11269404c20 Mon Sep 17 00:00:00 2001
From: Guillaume Lelarge <[email protected]>
Date: Tue, 12 Nov 2024 21:59:14 +0100
Subject: [PATCH v4] Enable BUFFERS by default with EXPLAIN ANALYZE
This automatically enables the BUFFER option when ANALYZE is used with
EXPLAIN. The BUFFER option stays disabled if ANALYZE isn't used.
The auto_explain extension gets the same treatment.
---
contrib/auto_explain/auto_explain.c | 4 +-
doc/src/sgml/auto-explain.sgml | 2 +-
doc/src/sgml/perform.sgml | 42 ++++++---
doc/src/sgml/ref/explain.sgml | 6 +-
src/backend/commands/explain.c | 7 ++
src/test/regress/expected/brin_multi.out | 18 ++--
src/test/regress/expected/explain.out | 36 ++++++-
.../regress/expected/incremental_sort.out | 4 +-
src/test/regress/expected/memoize.out | 2 +-
src/test/regress/expected/merge.out | 2 +-
src/test/regress/expected/partition_prune.out | 94 +++++++++----------
src/test/regress/expected/select.out | 2 +-
src/test/regress/expected/select_into.out | 4 +-
src/test/regress/expected/select_parallel.out | 6 +-
src/test/regress/expected/subselect.out | 2 +-
src/test/regress/expected/tidscan.out | 6 +-
src/test/regress/sql/brin_multi.sql | 18 ++--
src/test/regress/sql/explain.sql | 7 +-
src/test/regress/sql/incremental_sort.sql | 4 +-
src/test/regress/sql/memoize.sql | 2 +-
src/test/regress/sql/merge.sql | 2 +-
src/test/regress/sql/partition_prune.sql | 94 +++++++++----------
src/test/regress/sql/select.sql | 2 +-
src/test/regress/sql/select_into.sql | 4 +-
src/test/regress/sql/select_parallel.sql | 6 +-
src/test/regress/sql/subselect.sql | 2 +-
src/test/regress/sql/tidscan.sql | 6 +-
27 files changed, 220 insertions(+), 164 deletions(-)
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 623a674f99..484e009577 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -27,7 +27,7 @@ static int auto_explain_log_min_duration = -1; /* msec or -1 */
static int auto_explain_log_parameter_max_length = -1; /* bytes or -1 */
static bool auto_explain_log_analyze = false;
static bool auto_explain_log_verbose = false;
-static bool auto_explain_log_buffers = false;
+static bool auto_explain_log_buffers = true;
static bool auto_explain_log_wal = false;
static bool auto_explain_log_triggers = false;
static bool auto_explain_log_timing = true;
@@ -152,7 +152,7 @@ _PG_init(void)
"Log buffers usage.",
NULL,
&auto_explain_log_buffers,
- false,
+ true,
PGC_SUSET,
0,
NULL,
diff --git a/doc/src/sgml/auto-explain.sgml b/doc/src/sgml/auto-explain.sgml
index 0c4656ee30..6623d36c99 100644
--- a/doc/src/sgml/auto-explain.sgml
+++ b/doc/src/sgml/auto-explain.sgml
@@ -122,7 +122,7 @@ LOAD 'auto_explain';
equivalent to the <literal>BUFFERS</literal> option of <command>EXPLAIN</command>.
This parameter has no effect
unless <varname>auto_explain.log_analyze</varname> is enabled.
- This parameter is off by default.
+ This parameter is on by default.
Only superusers can change this setting.
</para>
</listitem>
diff --git a/doc/src/sgml/perform.sgml b/doc/src/sgml/perform.sgml
index cd12b9ce48..24916b82b5 100644
--- a/doc/src/sgml/perform.sgml
+++ b/doc/src/sgml/perform.sgml
@@ -722,13 +722,19 @@ WHERE t1.unique1 < 10 AND t1.unique2 = t2.unique2;
QUERY PLAN
-------------------------------------------------------------------&zwsp;--------------------------------------------------------------
Nested Loop (cost=4.65..118.50 rows=10 width=488) (actual time=0.017..0.051 rows=10 loops=1)
+ Buffers: shared hit=36 read=6
-> Bitmap Heap Scan on tenk1 t1 (cost=4.36..39.38 rows=10 width=244) (actual time=0.009..0.017 rows=10 loops=1)
Recheck Cond: (unique1 < 10)
Heap Blocks: exact=10
+ Buffers: shared hit=3 read=5 written=4
-> Bitmap Index Scan on tenk1_unique1 (cost=0.00..4.36 rows=10 width=0) (actual time=0.004..0.004 rows=10 loops=1)
Index Cond: (unique1 < 10)
+ Buffers: shared hit=2
-> Index Scan using tenk2_unique2 on tenk2 t2 (cost=0.29..7.90 rows=1 width=244) (actual time=0.003..0.003 rows=1 loops=10)
Index Cond: (unique2 = t1.unique2)
+ Buffers: shared hit=24 read=6
+ Planning:
+ Buffers: shared hit=15 dirtied=9
Planning Time: 0.485 ms
Execution Time: 0.073 ms
</screen>
@@ -769,16 +775,24 @@ WHERE t1.unique1 < 100 AND t1.unique2 = t2.unique2 ORDER BY t1.fivethous;
Sort (cost=713.05..713.30 rows=100 width=488) (actual time=2.995..3.002 rows=100 loops=1)
Sort Key: t1.fivethous
Sort Method: quicksort Memory: 74kB
+ Buffers: shared hit=440
-> Hash Join (cost=226.23..709.73 rows=100 width=488) (actual time=0.515..2.920 rows=100 loops=1)
Hash Cond: (t2.unique2 = t1.unique2)
+ Buffers: shared hit=437
-> Seq Scan on tenk2 t2 (cost=0.00..445.00 rows=10000 width=244) (actual time=0.026..1.790 rows=10000 loops=1)
+ Buffers: shared hit=345
-> Hash (cost=224.98..224.98 rows=100 width=244) (actual time=0.476..0.477 rows=100 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 35kB
+ Buffers: shared hit=92
-> Bitmap Heap Scan on tenk1 t1 (cost=5.06..224.98 rows=100 width=244) (actual time=0.030..0.450 rows=100 loops=1)
Recheck Cond: (unique1 < 100)
Heap Blocks: exact=90
+ Buffers: shared hit=92
-> Bitmap Index Scan on tenk1_unique1 (cost=0.00..5.04 rows=100 width=0) (actual time=0.013..0.013 rows=100 loops=1)
Index Cond: (unique1 < 100)
+ Buffers: shared hit=2
+ Planning:
+ Buffers: shared hit=12
Planning Time: 0.187 ms
Execution Time: 3.036 ms
</screen>
@@ -803,6 +817,7 @@ EXPLAIN ANALYZE SELECT * FROM tenk1 WHERE ten < 7;
Seq Scan on tenk1 (cost=0.00..470.00 rows=7000 width=244) (actual time=0.030..1.995 rows=7000 loops=1)
Filter: (ten < 7)
Rows Removed by Filter: 3000
+ Buffers: shared hit=345
Planning Time: 0.102 ms
Execution Time: 2.145 ms
</screen>
@@ -826,6 +841,7 @@ EXPLAIN ANALYZE SELECT * FROM polygon_tbl WHERE f1 @> polygon '(0.5,2.0)';
Seq Scan on polygon_tbl (cost=0.00..1.09 rows=1 width=85) (actual time=0.023..0.023 rows=0 loops=1)
Filter: (f1 @> '((0.5,2))'::polygon)
Rows Removed by Filter: 7
+ Buffers: shared hit=1
Planning Time: 0.039 ms
Execution Time: 0.033 ms
</screen>
@@ -845,6 +861,7 @@ EXPLAIN ANALYZE SELECT * FROM polygon_tbl WHERE f1 @> polygon '(0.5,2.0)';
Index Scan using gpolygonind on polygon_tbl (cost=0.13..8.15 rows=1 width=85) (actual time=0.074..0.074 rows=0 loops=1)
Index Cond: (f1 @> '((0.5,2))'::polygon)
Rows Removed by Index Recheck: 1
+ Buffers: shared hit=3
Planning Time: 0.039 ms
Execution Time: 0.098 ms
</screen>
@@ -857,34 +874,27 @@ EXPLAIN ANALYZE SELECT * FROM polygon_tbl WHERE f1 @> polygon '(0.5,2.0)';
</para>
<para>
- <command>EXPLAIN</command> has a <literal>BUFFERS</literal> option that can be used with
- <literal>ANALYZE</literal> to get even more run time statistics:
+ <command>EXPLAIN</command> has a <literal>BUFFERS</literal> option which is
+ enabled by default when <literal>ANALYZE</literal> is used. The numbers provided
+ by <literal>BUFFERS</literal> help to identify which parts of the query are the
+ most I/O-intensive. You can turn it off:
<screen>
-EXPLAIN (ANALYZE, BUFFERS) SELECT * FROM tenk1 WHERE unique1 < 100 AND unique2 > 9000;
+EXPLAIN (ANALYZE, BUFFERS OFF) SELECT * FROM tenk1 WHERE unique1 < 100 AND unique2 > 9000;
QUERY PLAN
-------------------------------------------------------------------&zwsp;--------------------------------------------------------------
Bitmap Heap Scan on tenk1 (cost=25.07..60.11 rows=10 width=244) (actual time=0.105..0.114 rows=10 loops=1)
Recheck Cond: ((unique1 < 100) AND (unique2 > 9000))
Heap Blocks: exact=10
- Buffers: shared hit=14 read=3
-> BitmapAnd (cost=25.07..25.07 rows=10 width=0) (actual time=0.100..0.101 rows=0 loops=1)
- Buffers: shared hit=4 read=3
-> Bitmap Index Scan on tenk1_unique1 (cost=0.00..5.04 rows=100 width=0) (actual time=0.027..0.027 rows=100 loops=1)
Index Cond: (unique1 < 100)
- Buffers: shared hit=2
-> Bitmap Index Scan on tenk1_unique2 (cost=0.00..19.78 rows=999 width=0) (actual time=0.070..0.070 rows=999 loops=1)
Index Cond: (unique2 > 9000)
- Buffers: shared hit=2 read=3
- Planning:
- Buffers: shared hit=3
Planning Time: 0.162 ms
Execution Time: 0.143 ms
</screen>
-
- The numbers provided by <literal>BUFFERS</literal> help to identify which parts
- of the query are the most I/O-intensive.
</para>
<para>
@@ -906,8 +916,12 @@ EXPLAIN ANALYZE UPDATE tenk1 SET hundred = hundred + 1 WHERE unique1 < 100;
-> Bitmap Heap Scan on tenk1 (cost=5.06..225.23 rows=100 width=10) (actual time=0.065..0.141 rows=100 loops=1)
Recheck Cond: (unique1 < 100)
Heap Blocks: exact=90
+ Buffers: shared hit=92
-> Bitmap Index Scan on tenk1_unique1 (cost=0.00..5.04 rows=100 width=0) (actual time=0.031..0.031 rows=100 loops=1)
Index Cond: (unique1 < 100)
+ Buffers: shared hit=2
+ Planning:
+ Buffers: shared hit=91
Planning Time: 0.151 ms
Execution Time: 1.856 ms
@@ -1040,10 +1054,14 @@ EXPLAIN ANALYZE SELECT * FROM tenk1 WHERE unique1 < 100 AND unique2 > 9000
QUERY PLAN
-------------------------------------------------------------------&zwsp;------------------------------------------------------------
Limit (cost=0.29..14.33 rows=2 width=244) (actual time=0.051..0.071 rows=2 loops=1)
+ Buffers: shared hit=16
-> Index Scan using tenk1_unique2 on tenk1 (cost=0.29..70.50 rows=10 width=244) (actual time=0.051..0.070 rows=2 loops=1)
Index Cond: (unique2 > 9000)
Filter: (unique1 < 100)
Rows Removed by Filter: 287
+ Buffers: shared hit=16
+ Planning:
+ Buffers: shared hit=49
Planning Time: 0.077 ms
Execution Time: 0.086 ms
</screen>
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index db9d3a8549..4d6f655b03 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -201,8 +201,10 @@ ROLLBACK;
query processing.
The number of blocks shown for an
upper-level node includes those used by all its child nodes. In text
- format, only non-zero values are printed. This parameter defaults to
- <literal>FALSE</literal>.
+ format, only non-zero values are printed.
+ Buffers information is included by default when
+ <literal>ANALYZE</literal> is used but otherwise is not included by
+ default, but can be enabled using this option.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 7c0fd63b2f..d6815e01d1 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -198,6 +198,7 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
List *rewritten;
ListCell *lc;
bool timing_set = false;
+ bool buffers_set = false;
bool summary_set = false;
/* Parse options list. */
@@ -212,7 +213,10 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
else if (strcmp(opt->defname, "costs") == 0)
es->costs = defGetBoolean(opt);
else if (strcmp(opt->defname, "buffers") == 0)
+ {
+ buffers_set = true;
es->buffers = defGetBoolean(opt);
+ }
else if (strcmp(opt->defname, "wal") == 0)
es->wal = defGetBoolean(opt);
else if (strcmp(opt->defname, "settings") == 0)
@@ -292,6 +296,9 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
/* if the timing was not set explicitly, set default value */
es->timing = (timing_set) ? es->timing : es->analyze;
+ /* if the buffers was not set explicitly, set default value */
+ es->buffers = (buffers_set) ? es->buffers : es->analyze;
+
/* check that timing is used with EXPLAIN ANALYZE */
if (es->timing && !es->analyze)
ereport(ERROR,
diff --git a/src/test/regress/expected/brin_multi.out b/src/test/regress/expected/brin_multi.out
index ae9ce9d8ec..f2d1465818 100644
--- a/src/test/regress/expected/brin_multi.out
+++ b/src/test/regress/expected/brin_multi.out
@@ -845,7 +845,7 @@ INSERT INTO brin_date_test SELECT '5874897-12-01'::date + i FROM generate_series
CREATE INDEX ON brin_date_test USING brin (a date_minmax_multi_ops) WITH (pages_per_range=1);
SET enable_seqscan = off;
-- make sure the ranges were built correctly and 2023-01-01 eliminates all
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_date_test WHERE a = '2023-01-01'::date;
QUERY PLAN
-------------------------------------------------------------------------
@@ -864,7 +864,7 @@ INSERT INTO brin_timestamp_test
SELECT i FROM generate_series('2000-01-01'::timestamp, '2000-02-09'::timestamp, '1 day'::interval) s(i);
CREATE INDEX ON brin_timestamp_test USING brin (a timestamp_minmax_multi_ops) WITH (pages_per_range=1);
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_timestamp_test WHERE a = '2023-01-01'::timestamp;
QUERY PLAN
------------------------------------------------------------------------------
@@ -874,7 +874,7 @@ SELECT * FROM brin_timestamp_test WHERE a = '2023-01-01'::timestamp;
Index Cond: (a = '2023-01-01 00:00:00'::timestamp without time zone)
(4 rows)
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_timestamp_test WHERE a = '1900-01-01'::timestamp;
QUERY PLAN
------------------------------------------------------------------------------
@@ -892,7 +892,7 @@ INSERT INTO brin_date_test VALUES ('-infinity'), ('infinity');
INSERT INTO brin_date_test SELECT '2000-01-01'::date + i FROM generate_series(1, 40) s(i);
CREATE INDEX ON brin_date_test USING brin (a date_minmax_multi_ops) WITH (pages_per_range=1);
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_date_test WHERE a = '2023-01-01'::date;
QUERY PLAN
-------------------------------------------------------------------------
@@ -902,7 +902,7 @@ SELECT * FROM brin_date_test WHERE a = '2023-01-01'::date;
Index Cond: (a = '2023-01-01'::date)
(4 rows)
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_date_test WHERE a = '1900-01-01'::date;
QUERY PLAN
-------------------------------------------------------------------------
@@ -921,7 +921,7 @@ INSERT INTO brin_interval_test SELECT (i || ' years')::interval FROM generate_se
INSERT INTO brin_interval_test SELECT (i || ' years')::interval FROM generate_series( 177999980, 178000000) s(i);
CREATE INDEX ON brin_interval_test USING brin (a interval_minmax_multi_ops) WITH (pages_per_range=1);
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '-30 years'::interval;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -931,7 +931,7 @@ SELECT * FROM brin_interval_test WHERE a = '-30 years'::interval;
Index Cond: (a = '@ 30 years ago'::interval)
(4 rows)
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '30 years'::interval;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -949,7 +949,7 @@ INSERT INTO brin_interval_test VALUES ('-infinity'), ('infinity');
INSERT INTO brin_interval_test SELECT (i || ' days')::interval FROM generate_series(100, 140) s(i);
CREATE INDEX ON brin_interval_test USING brin (a interval_minmax_multi_ops) WITH (pages_per_range=1);
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '-30 years'::interval;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -959,7 +959,7 @@ SELECT * FROM brin_interval_test WHERE a = '-30 years'::interval;
Index Cond: (a = '@ 30 years ago'::interval)
(4 rows)
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '30 years'::interval;
QUERY PLAN
-----------------------------------------------------------------------------
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index d2eef8097c..930b4c6f48 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -77,7 +77,7 @@ select explain_filter('explain (analyze, verbose) select * from int8_tbl i8');
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (analyze, format text) select * from int8_tbl i8');
explain_filter
-----------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N loops=N)
@@ -85,7 +85,7 @@ select explain_filter('explain (analyze, buffers, format text) select * from int
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, format xml) select * from int8_tbl i8');
explain_filter
--------------------------------------------------------
<explain xmlns="http://www.postgresql.org/N/explain"> +
@@ -136,7 +136,15 @@ select explain_filter('explain (analyze, buffers, format xml) select * from int8
</explain>
(1 row)
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, format text) select * from int8_tbl i8');
+ explain_filter
+-----------------------------------------------------------------------------------------------
+ Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N loops=N)
+ Planning Time: N.N ms
+ Execution Time: N.N ms
+(3 rows)
+
+select explain_filter('explain (analyze, serialize, format yaml) select * from int8_tbl i8');
explain_filter
-------------------------------
- Plan: +
@@ -400,9 +408,29 @@ select explain_filter('explain (memory, analyze, format json) select * from int8
"Actual Total Time": N.N, +
"Actual Rows": N, +
"Actual Loops": N, +
- "Disabled": false +
+ "Disabled": false, +
+ "Shared Hit Blocks": N, +
+ "Shared Read Blocks": N, +
+ "Shared Dirtied Blocks": N, +
+ "Shared Written Blocks": N, +
+ "Local Hit Blocks": N, +
+ "Local Read Blocks": N, +
+ "Local Dirtied Blocks": N, +
+ "Local Written Blocks": N, +
+ "Temp Read Blocks": N, +
+ "Temp Written Blocks": N +
}, +
"Planning": { +
+ "Shared Hit Blocks": N, +
+ "Shared Read Blocks": N, +
+ "Shared Dirtied Blocks": N, +
+ "Shared Written Blocks": N, +
+ "Local Hit Blocks": N, +
+ "Local Read Blocks": N, +
+ "Local Dirtied Blocks": N, +
+ "Local Written Blocks": N, +
+ "Temp Read Blocks": N, +
+ "Temp Written Blocks": N, +
"Memory Used": N, +
"Memory Allocated": N +
}, +
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 2df7a5db12..d597575840 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -39,7 +39,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -55,7 +55,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/expected/memoize.out b/src/test/regress/expected/memoize.out
index f6b8329cd6..5ecf971dad 100644
--- a/src/test/regress/expected/memoize.out
+++ b/src/test/regress/expected/memoize.out
@@ -10,7 +10,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/expected/merge.out b/src/test/regress/expected/merge.out
index 521d70a891..28d8551063 100644
--- a/src/test/regress/expected/merge.out
+++ b/src/test/regress/expected/merge.out
@@ -1621,7 +1621,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 7a03b4e360..c710f316d7 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
@@ -2127,7 +2127,7 @@ create table ab_a3_b3 partition of ab_a3 for values in (3);
set enable_indexonlyscan = off;
prepare ab_q1 (int, int, int) as
select * from ab where a between $1 and $2 and b <= $3;
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2, 3);
QUERY PLAN
---------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2140,7 +2140,7 @@ explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 2, 3);
Filter: ((a >= $1) AND (a <= $2) AND (b <= $3))
(8 rows)
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (1, 2, 3);
QUERY PLAN
---------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2163,7 +2163,7 @@ deallocate ab_q1;
-- Runtime pruning after optimizer pruning
prepare ab_q1 (int, int) as
select a from ab where a between $1 and $2 and b < 3;
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2);
QUERY PLAN
---------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2174,7 +2174,7 @@ explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 2);
Filter: ((a >= $1) AND (a <= $2) AND (b < 3))
(6 rows)
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 4);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 4);
QUERY PLAN
---------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2193,7 +2193,7 @@ explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 4);
-- different levels of partitioning.
prepare ab_q2 (int, int) as
select a from ab where a between $1 and $2 and b < (select 3);
-explain (analyze, costs off, summary off, timing off) execute ab_q2 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q2 (2, 2);
QUERY PLAN
-----------------------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2211,7 +2211,7 @@ explain (analyze, costs off, summary off, timing off) execute ab_q2 (2, 2);
-- As above, but swap the PARAM_EXEC Param to the first partition level
prepare ab_q3 (int, int) as
select a from ab where b between $1 and $2 and a < (select 3);
-explain (analyze, costs off, summary off, timing off) execute ab_q3 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q3 (2, 2);
QUERY PLAN
-----------------------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2273,7 +2273,7 @@ begin;
-- Test run-time pruning using stable functions
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
QUERY PLAN
------------------------------------------------------------------
Append (actual rows=1 loops=1)
@@ -2283,7 +2283,7 @@ explain (analyze, costs off, summary off, timing off) select * from list_part wh
(4 rows)
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
QUERY PLAN
------------------------------------------------------------------
Append (actual rows=4 loops=1)
@@ -2298,7 +2298,7 @@ explain (analyze, costs off, summary off, timing off) select * from list_part wh
(9 rows)
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
QUERY PLAN
------------------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2334,7 +2334,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -2641,7 +2641,7 @@ reset parallel_tuple_cost;
reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
QUERY PLAN
-------------------------------------------------------------------------
@@ -2700,7 +2700,7 @@ select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1
(52 rows)
-- Test run-time partition pruning with UNION ALL parents
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from (select * from ab where a = 1 union all select * from ab) ab where b = (select 1);
QUERY PLAN
-------------------------------------------------------------------------------
@@ -2744,7 +2744,7 @@ select * from (select * from ab where a = 1 union all select * from ab) ab where
(37 rows)
-- A case containing a UNION ALL with a non-partitioned child.
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from (select * from ab where a = 1 union all (values(10,5)) union all select * from ab) ab where b = (select 1);
QUERY PLAN
-------------------------------------------------------------------------------
@@ -2803,7 +2803,7 @@ union all
select tableoid::regclass,a,b from ab
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
QUERY PLAN
--------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2952,7 +2952,7 @@ create index tprt6_idx on tprt_6 (col1);
insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
--------------------------------------------------------------------------
@@ -2973,7 +2973,7 @@ select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
Index Cond: (col1 < tbl1.col1)
(15 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
--------------------------------------------------------------------------
@@ -3018,7 +3018,7 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
--------------------------------------------------------------------------
@@ -3039,7 +3039,7 @@ select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
Index Cond: (col1 < tbl1.col1)
(15 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
--------------------------------------------------------------------------
@@ -3103,7 +3103,7 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
QUERY PLAN
--------------------------------------------------------------------------
@@ -3135,7 +3135,7 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-------------------------------------------------------------------
@@ -3175,7 +3175,7 @@ alter table part_cab attach partition part_abc_p1 for values in(3);
prepare part_abc_q1 (int, int, int) as
select * from part_abc where a = $1 and b = $2 and c = $3;
-- Single partition should be scanned.
-explain (analyze, costs off, summary off, timing off) execute part_abc_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off) execute part_abc_q1 (1, 2, 3);
QUERY PLAN
----------------------------------------------------------
Seq Scan on part_abc_p1 part_abc (actual rows=0 loops=1)
@@ -3200,7 +3200,7 @@ select * from listp where b = 1;
-- partitions before finally detecting the correct set of 2nd level partitions
-- which match the given parameter.
prepare q1 (int,int) as select * from listp where b in ($1,$2);
-explain (analyze, costs off, summary off, timing off) execute q1 (1,1);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,1);
QUERY PLAN
-------------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -3209,7 +3209,7 @@ explain (analyze, costs off, summary off, timing off) execute q1 (1,1);
Filter: (b = ANY (ARRAY[$1, $2]))
(4 rows)
-explain (analyze, costs off, summary off, timing off) execute q1 (2,2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (2,2);
QUERY PLAN
-------------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -3219,7 +3219,7 @@ explain (analyze, costs off, summary off, timing off) execute q1 (2,2);
(4 rows)
-- Try with no matching partitions.
-explain (analyze, costs off, summary off, timing off) execute q1 (0,0);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (0,0);
QUERY PLAN
--------------------------------
Append (actual rows=0 loops=1)
@@ -3230,7 +3230,7 @@ deallocate q1;
-- Test more complex cases where a not-equal condition further eliminates partitions.
prepare q1 (int,int,int,int) as select * from listp where b in($1,$2) and $3 <> b and $4 <> b;
-- Both partitions allowed by IN clause, but one disallowed by <> clause
-explain (analyze, costs off, summary off, timing off) execute q1 (1,2,2,0);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,0);
QUERY PLAN
-------------------------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -3240,7 +3240,7 @@ explain (analyze, costs off, summary off, timing off) execute q1 (1,2,2,0);
(4 rows)
-- Both partitions allowed by IN clause, then both excluded again by <> clauses.
-explain (analyze, costs off, summary off, timing off) execute q1 (1,2,2,1);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,1);
QUERY PLAN
--------------------------------
Append (actual rows=0 loops=1)
@@ -3248,7 +3248,7 @@ explain (analyze, costs off, summary off, timing off) execute q1 (1,2,2,1);
(2 rows)
-- Ensure Params that evaluate to NULL properly prune away all partitions
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from listp where a = (select null::int);
QUERY PLAN
------------------------------------------------------
@@ -3273,7 +3273,7 @@ create table stable_qual_pruning2 partition of stable_qual_pruning
create table stable_qual_pruning3 partition of stable_qual_pruning
for values from ('3000-02-01') to ('3000-03-01');
-- comparison against a stable value requires run-time pruning
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning where a < localtimestamp;
QUERY PLAN
--------------------------------------------------------------------------------------
@@ -3286,7 +3286,7 @@ select * from stable_qual_pruning where a < localtimestamp;
(6 rows)
-- timestamp < timestamptz comparison is only stable, not immutable
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning where a < '2000-02-01'::timestamptz;
QUERY PLAN
--------------------------------------------------------------------------------------
@@ -3297,7 +3297,7 @@ select * from stable_qual_pruning where a < '2000-02-01'::timestamptz;
(4 rows)
-- check ScalarArrayOp cases
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamp[]);
QUERY PLAN
@@ -3306,7 +3306,7 @@ select * from stable_qual_pruning
One-Time Filter: false
(2 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamp[]);
QUERY PLAN
@@ -3315,7 +3315,7 @@ select * from stable_qual_pruning
Filter: (a = ANY ('{"Tue Feb 01 00:00:00 2000","Fri Jan 01 00:00:00 2010"}'::timestamp without time zone[]))
(2 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', localtimestamp]::timestamp[]);
QUERY PLAN
@@ -3326,7 +3326,7 @@ select * from stable_qual_pruning
Filter: (a = ANY (ARRAY['Tue Feb 01 00:00:00 2000'::timestamp without time zone, LOCALTIMESTAMP]))
(4 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamptz[]);
QUERY PLAN
@@ -3335,7 +3335,7 @@ select * from stable_qual_pruning
Subplans Removed: 3
(2 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamptz[]);
QUERY PLAN
@@ -3346,7 +3346,7 @@ select * from stable_qual_pruning
Filter: (a = ANY ('{"Tue Feb 01 00:00:00 2000 PST","Fri Jan 01 00:00:00 2010 PST"}'::timestamp with time zone[]))
(4 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(null::timestamptz[]);
QUERY PLAN
@@ -3374,7 +3374,7 @@ create table mc3p1 partition of mc3p
create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from mc3p where a < 3 and abs(b) = 1;
QUERY PLAN
--------------------------------------------------------
@@ -3394,7 +3394,7 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
execute ps1(1);
QUERY PLAN
-------------------------------------------------------------
@@ -3409,7 +3409,7 @@ execute ps1(1);
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
execute ps2(1);
QUERY PLAN
--------------------------------------------------------------
@@ -3431,7 +3431,7 @@ insert into boolvalues values('t'),('f');
create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from boolp where a = (select value from boolvalues where value);
QUERY PLAN
-----------------------------------------------------------
@@ -3446,7 +3446,7 @@ select * from boolp where a = (select value from boolvalues where value);
Filter: (a = (InitPlan 1).col1)
(9 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from boolp where a = (select value from boolvalues where not value);
QUERY PLAN
-----------------------------------------------------------
@@ -3475,7 +3475,7 @@ insert into ma_test select x,x from generate_series(0,29) t(x);
create index on ma_test (b);
analyze ma_test;
prepare mt_q1 (int) as select a from ma_test where a >= $1 and a % 10 = 5 order by b;
-explain (analyze, costs off, summary off, timing off) execute mt_q1(15);
+explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(15);
QUERY PLAN
-----------------------------------------------------------------------------------------
Merge Append (actual rows=2 loops=1)
@@ -3496,7 +3496,7 @@ execute mt_q1(15);
25
(2 rows)
-explain (analyze, costs off, summary off, timing off) execute mt_q1(25);
+explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(25);
QUERY PLAN
-----------------------------------------------------------------------------------------
Merge Append (actual rows=1 loops=1)
@@ -3514,7 +3514,7 @@ execute mt_q1(25);
(1 row)
-- Ensure MergeAppend behaves correctly when no subplans match
-explain (analyze, costs off, summary off, timing off) execute mt_q1(35);
+explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(35);
QUERY PLAN
--------------------------------------
Merge Append (actual rows=0 loops=1)
@@ -3542,7 +3542,7 @@ explain (analyze, verbose, costs off, summary off, timing off) execute mt_q2 (35
deallocate mt_q2;
-- ensure initplan params properly prune partitions
-explain (analyze, costs off, summary off, timing off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
+explain (analyze, costs off, summary off, timing off, buffers off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
QUERY PLAN
-----------------------------------------------------------------------------------------------
Merge Append (actual rows=20 loops=1)
@@ -3992,7 +3992,7 @@ create table listp (a int, b int) partition by list (a);
create table listp1 partition of listp for values in(1);
create table listp2 partition of listp for values in(2) partition by list(b);
create table listp2_10 partition of listp2 for values in (10);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from listp where a = (select 2) and b <> 10;
QUERY PLAN
---------------------------------------------------
@@ -4117,7 +4117,7 @@ create table rangep_0_to_100_3 partition of rangep_0_to_100 for values in(3);
create table rangep_100_to_200 partition of rangep for values from (100) to (200);
create index on rangep (a);
-- Ensure run-time pruning works on the nested Merge Append
-explain (analyze on, costs off, timing off, summary off)
+explain (analyze on, costs off, timing off, summary off, buffers off)
select * from rangep where b IN((select 1),(select 2)) order by a;
QUERY PLAN
------------------------------------------------------------------------------------------------------------
diff --git a/src/test/regress/expected/select.out b/src/test/regress/expected/select.out
index 33a6dceb0e..88911ca2b9 100644
--- a/src/test/regress/expected/select.out
+++ b/src/test/regress/expected/select.out
@@ -757,7 +757,7 @@ select * from onek2 where unique2 = 11 and stringu1 = 'ATAAAA';
(1 row)
-- actually run the query with an analyze to use the partial index
-explain (costs off, analyze on, timing off, summary off)
+explain (costs off, analyze on, timing off, summary off, buffers off)
select * from onek2 where unique2 = 11 and stringu1 = 'ATAAAA';
QUERY PLAN
-----------------------------------------------------------------
diff --git a/src/test/regress/expected/select_into.out b/src/test/regress/expected/select_into.out
index b79fe9a1c0..d6578d992c 100644
--- a/src/test/regress/expected/select_into.out
+++ b/src/test/regress/expected/select_into.out
@@ -25,7 +25,7 @@ CREATE TABLE selinto_schema.tbl_withdata1 (a)
AS SELECT generate_series(1,3) WITH DATA;
INSERT INTO selinto_schema.tbl_withdata1 VALUES (4);
ERROR: permission denied for table tbl_withdata1
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
CREATE TABLE selinto_schema.tbl_withdata2 (a) AS
SELECT generate_series(1,3) WITH DATA;
QUERY PLAN
@@ -62,7 +62,7 @@ EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
-- EXECUTE and WITH NO DATA, passes.
CREATE TABLE selinto_schema.tbl_nodata3 (a) AS
EXECUTE data_sel WITH NO DATA;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
CREATE TABLE selinto_schema.tbl_nodata4 (a) AS
EXECUTE data_sel WITH NO DATA;
QUERY PLAN
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 8c31f6460d..a809036453 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -580,7 +580,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off)
+explain (analyze, timing off, summary off, costs off, buffers off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
QUERY PLAN
@@ -606,7 +606,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off)
+ explain (analyze, timing off, summary off, costs off, buffers off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -1169,7 +1169,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
QUERY PLAN
-------------------------------------------------------------
Gather (actual rows=10000 loops=1)
diff --git a/src/test/regress/expected/subselect.out b/src/test/regress/expected/subselect.out
index 2d35de3fad..e04cea3d36 100644
--- a/src/test/regress/expected/subselect.out
+++ b/src/test/regress/expected/subselect.out
@@ -1675,7 +1675,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off)
+ explain (analyze, summary off, timing off, costs off, buffers off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
diff --git a/src/test/regress/expected/tidscan.out b/src/test/regress/expected/tidscan.out
index f133b5a4ac..f6ebdf0601 100644
--- a/src/test/regress/expected/tidscan.out
+++ b/src/test/regress/expected/tidscan.out
@@ -189,7 +189,7 @@ FETCH NEXT FROM c;
(1 row)
-- perform update
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
UPDATE tidscan SET id = -id WHERE CURRENT OF c RETURNING *;
QUERY PLAN
---------------------------------------------------
@@ -205,7 +205,7 @@ FETCH NEXT FROM c;
(1 row)
-- perform update
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
UPDATE tidscan SET id = -id WHERE CURRENT OF c RETURNING *;
QUERY PLAN
---------------------------------------------------
@@ -229,7 +229,7 @@ FETCH NEXT FROM c;
(0 rows)
-- should error out
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
UPDATE tidscan SET id = -id WHERE CURRENT OF c RETURNING *;
ERROR: cursor "c" is not positioned on a row
ROLLBACK;
diff --git a/src/test/regress/sql/brin_multi.sql b/src/test/regress/sql/brin_multi.sql
index 55349b4e1f..dda9fb73e0 100644
--- a/src/test/regress/sql/brin_multi.sql
+++ b/src/test/regress/sql/brin_multi.sql
@@ -619,7 +619,7 @@ CREATE INDEX ON brin_date_test USING brin (a date_minmax_multi_ops) WITH (pages_
SET enable_seqscan = off;
-- make sure the ranges were built correctly and 2023-01-01 eliminates all
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_date_test WHERE a = '2023-01-01'::date;
DROP TABLE brin_date_test;
@@ -636,10 +636,10 @@ CREATE INDEX ON brin_timestamp_test USING brin (a timestamp_minmax_multi_ops) WI
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_timestamp_test WHERE a = '2023-01-01'::timestamp;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_timestamp_test WHERE a = '1900-01-01'::timestamp;
DROP TABLE brin_timestamp_test;
@@ -655,10 +655,10 @@ CREATE INDEX ON brin_date_test USING brin (a date_minmax_multi_ops) WITH (pages_
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_date_test WHERE a = '2023-01-01'::date;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_date_test WHERE a = '1900-01-01'::date;
DROP TABLE brin_date_test;
@@ -676,10 +676,10 @@ CREATE INDEX ON brin_interval_test USING brin (a interval_minmax_multi_ops) WITH
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '-30 years'::interval;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '30 years'::interval;
DROP TABLE brin_interval_test;
@@ -695,10 +695,10 @@ CREATE INDEX ON brin_interval_test USING brin (a interval_minmax_multi_ops) WITH
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '-30 years'::interval;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '30 years'::interval;
DROP TABLE brin_interval_test;
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index 3ca285a1d7..3f41396886 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -64,9 +64,10 @@ set track_io_timing = off;
select explain_filter('explain select * from int8_tbl i8');
select explain_filter('explain (analyze) select * from int8_tbl i8');
select explain_filter('explain (analyze, verbose) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, format text) select * from int8_tbl i8');
+select explain_filter('explain (analyze, format xml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, format text) select * from int8_tbl i8');
+select explain_filter('explain (analyze, serialize, format yaml) select * from int8_tbl i8');
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
diff --git a/src/test/regress/sql/incremental_sort.sql b/src/test/regress/sql/incremental_sort.sql
index 98b20e17e1..f1f8fae565 100644
--- a/src/test/regress/sql/incremental_sort.sql
+++ b/src/test/regress/sql/incremental_sort.sql
@@ -21,7 +21,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -38,7 +38,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/sql/memoize.sql b/src/test/regress/sql/memoize.sql
index 2eaeb1477a..d5aab4e566 100644
--- a/src/test/regress/sql/memoize.sql
+++ b/src/test/regress/sql/memoize.sql
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/sql/merge.sql b/src/test/regress/sql/merge.sql
index 5ddcca84f8..54929a92fa 100644
--- a/src/test/regress/sql/merge.sql
+++ b/src/test/regress/sql/merge.sql
@@ -1072,7 +1072,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index 442428d937..8cd4a21fdc 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -12,7 +12,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
@@ -465,8 +465,8 @@ set enable_indexonlyscan = off;
prepare ab_q1 (int, int, int) as
select * from ab where a between $1 and $2 and b <= $3;
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 2, 3);
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (1, 2, 3);
deallocate ab_q1;
@@ -474,21 +474,21 @@ deallocate ab_q1;
prepare ab_q1 (int, int) as
select a from ab where a between $1 and $2 and b < 3;
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 2);
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 4);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 4);
-- Ensure a mix of PARAM_EXTERN and PARAM_EXEC Params work together at
-- different levels of partitioning.
prepare ab_q2 (int, int) as
select a from ab where a between $1 and $2 and b < (select 3);
-explain (analyze, costs off, summary off, timing off) execute ab_q2 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q2 (2, 2);
-- As above, but swap the PARAM_EXEC Param to the first partition level
prepare ab_q3 (int, int) as
select a from ab where b between $1 and $2 and a < (select 3);
-explain (analyze, costs off, summary off, timing off) execute ab_q3 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q3 (2, 2);
--
-- Test runtime pruning with hash partitioned tables
@@ -538,13 +538,13 @@ begin;
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
rollback;
@@ -567,7 +567,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -650,15 +650,15 @@ reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
-- Test run-time partition pruning with UNION ALL parents
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from (select * from ab where a = 1 union all select * from ab) ab where b = (select 1);
-- A case containing a UNION ALL with a non-partitioned child.
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from (select * from ab where a = 1 union all (values(10,5)) union all select * from ab) ab where b = (select 1);
-- Another UNION ALL test, but containing a mix of exec init and exec run-time pruning.
@@ -678,7 +678,7 @@ union all
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
-- Ensure we see just the xy_1 row.
execute ab_q6(100);
@@ -733,10 +733,10 @@ insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -749,10 +749,10 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -766,7 +766,7 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -776,7 +776,7 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -799,7 +799,7 @@ prepare part_abc_q1 (int, int, int) as
select * from part_abc where a = $1 and b = $2 and c = $3;
-- Single partition should be scanned.
-explain (analyze, costs off, summary off, timing off) execute part_abc_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off) execute part_abc_q1 (1, 2, 3);
deallocate part_abc_q1;
@@ -819,12 +819,12 @@ select * from listp where b = 1;
-- which match the given parameter.
prepare q1 (int,int) as select * from listp where b in ($1,$2);
-explain (analyze, costs off, summary off, timing off) execute q1 (1,1);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,1);
-explain (analyze, costs off, summary off, timing off) execute q1 (2,2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (2,2);
-- Try with no matching partitions.
-explain (analyze, costs off, summary off, timing off) execute q1 (0,0);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (0,0);
deallocate q1;
@@ -832,13 +832,13 @@ deallocate q1;
prepare q1 (int,int,int,int) as select * from listp where b in($1,$2) and $3 <> b and $4 <> b;
-- Both partitions allowed by IN clause, but one disallowed by <> clause
-explain (analyze, costs off, summary off, timing off) execute q1 (1,2,2,0);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,0);
-- Both partitions allowed by IN clause, then both excluded again by <> clauses.
-explain (analyze, costs off, summary off, timing off) execute q1 (1,2,2,1);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,1);
-- Ensure Params that evaluate to NULL properly prune away all partitions
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from listp where a = (select null::int);
drop table listp;
@@ -855,30 +855,30 @@ create table stable_qual_pruning3 partition of stable_qual_pruning
for values from ('3000-02-01') to ('3000-03-01');
-- comparison against a stable value requires run-time pruning
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning where a < localtimestamp;
-- timestamp < timestamptz comparison is only stable, not immutable
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning where a < '2000-02-01'::timestamptz;
-- check ScalarArrayOp cases
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamp[]);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamp[]);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', localtimestamp]::timestamp[]);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamptz[]);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamptz[]);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(null::timestamptz[]);
@@ -898,7 +898,7 @@ create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from mc3p where a < 3 and abs(b) = 1;
--
@@ -908,12 +908,12 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
execute ps1(1);
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
execute ps2(1);
deallocate ps2;
@@ -927,10 +927,10 @@ create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from boolp where a = (select value from boolvalues where value);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from boolp where a = (select value from boolvalues where not value);
drop table boolp;
@@ -950,12 +950,12 @@ create index on ma_test (b);
analyze ma_test;
prepare mt_q1 (int) as select a from ma_test where a >= $1 and a % 10 = 5 order by b;
-explain (analyze, costs off, summary off, timing off) execute mt_q1(15);
+explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(15);
execute mt_q1(15);
-explain (analyze, costs off, summary off, timing off) execute mt_q1(25);
+explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(25);
execute mt_q1(25);
-- Ensure MergeAppend behaves correctly when no subplans match
-explain (analyze, costs off, summary off, timing off) execute mt_q1(35);
+explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(35);
execute mt_q1(35);
deallocate mt_q1;
@@ -968,7 +968,7 @@ explain (analyze, verbose, costs off, summary off, timing off) execute mt_q2 (35
deallocate mt_q2;
-- ensure initplan params properly prune partitions
-explain (analyze, costs off, summary off, timing off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
+explain (analyze, costs off, summary off, timing off, buffers off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
reset enable_seqscan;
reset enable_sort;
@@ -1148,7 +1148,7 @@ create table listp1 partition of listp for values in(1);
create table listp2 partition of listp for values in(2) partition by list(b);
create table listp2_10 partition of listp2 for values in (10);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from listp where a = (select 2) and b <> 10;
--
@@ -1216,7 +1216,7 @@ create table rangep_100_to_200 partition of rangep for values from (100) to (200
create index on rangep (a);
-- Ensure run-time pruning works on the nested Merge Append
-explain (analyze on, costs off, timing off, summary off)
+explain (analyze on, costs off, timing off, summary off, buffers off)
select * from rangep where b IN((select 1),(select 2)) order by a;
reset enable_sort;
drop table rangep;
diff --git a/src/test/regress/sql/select.sql b/src/test/regress/sql/select.sql
index 019f1e7673..1d1bf2b931 100644
--- a/src/test/regress/sql/select.sql
+++ b/src/test/regress/sql/select.sql
@@ -196,7 +196,7 @@ explain (costs off)
select * from onek2 where unique2 = 11 and stringu1 = 'ATAAAA';
select * from onek2 where unique2 = 11 and stringu1 = 'ATAAAA';
-- actually run the query with an analyze to use the partial index
-explain (costs off, analyze on, timing off, summary off)
+explain (costs off, analyze on, timing off, summary off, buffers off)
select * from onek2 where unique2 = 11 and stringu1 = 'ATAAAA';
explain (costs off)
select unique2 from onek2 where unique2 = 11 and stringu1 = 'ATAAAA';
diff --git a/src/test/regress/sql/select_into.sql b/src/test/regress/sql/select_into.sql
index 689c448cc2..106cdde187 100644
--- a/src/test/regress/sql/select_into.sql
+++ b/src/test/regress/sql/select_into.sql
@@ -30,7 +30,7 @@ SET SESSION AUTHORIZATION regress_selinto_user;
CREATE TABLE selinto_schema.tbl_withdata1 (a)
AS SELECT generate_series(1,3) WITH DATA;
INSERT INTO selinto_schema.tbl_withdata1 VALUES (4);
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
CREATE TABLE selinto_schema.tbl_withdata2 (a) AS
SELECT generate_series(1,3) WITH DATA;
-- WITH NO DATA, passes.
@@ -49,7 +49,7 @@ EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
-- EXECUTE and WITH NO DATA, passes.
CREATE TABLE selinto_schema.tbl_nodata3 (a) AS
EXECUTE data_sel WITH NO DATA;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
CREATE TABLE selinto_schema.tbl_nodata4 (a) AS
EXECUTE data_sel WITH NO DATA;
RESET SESSION AUTHORIZATION;
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 5b4a6e1088..71a75bc86e 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -230,7 +230,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off)
+explain (analyze, timing off, summary off, costs off, buffers off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
alter table tenk2 reset (parallel_workers);
@@ -242,7 +242,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off)
+ explain (analyze, timing off, summary off, costs off, buffers off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -450,7 +450,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
ROLLBACK TO SAVEPOINT settings;
-- provoke error in worker
diff --git a/src/test/regress/sql/subselect.sql b/src/test/regress/sql/subselect.sql
index af6e157aca..c53c7f724c 100644
--- a/src/test/regress/sql/subselect.sql
+++ b/src/test/regress/sql/subselect.sql
@@ -857,7 +857,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off)
+ explain (analyze, summary off, timing off, costs off, buffers off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
diff --git a/src/test/regress/sql/tidscan.sql b/src/test/regress/sql/tidscan.sql
index 313e0fb9b6..1b82d5f1a5 100644
--- a/src/test/regress/sql/tidscan.sql
+++ b/src/test/regress/sql/tidscan.sql
@@ -68,17 +68,17 @@ DECLARE c CURSOR FOR SELECT ctid, * FROM tidscan;
FETCH NEXT FROM c; -- skip one row
FETCH NEXT FROM c;
-- perform update
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
UPDATE tidscan SET id = -id WHERE CURRENT OF c RETURNING *;
FETCH NEXT FROM c;
-- perform update
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
UPDATE tidscan SET id = -id WHERE CURRENT OF c RETURNING *;
SELECT * FROM tidscan;
-- position cursor past any rows
FETCH NEXT FROM c;
-- should error out
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
UPDATE tidscan SET id = -id WHERE CURRENT OF c RETURNING *;
ROLLBACK;
--
2.47.0
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-11-20 21:22 David Rowley <[email protected]>
parent: Guillaume Lelarge <[email protected]>
1 sibling, 4 replies; 32+ messages in thread
From: David Rowley @ 2024-11-20 21:22 UTC (permalink / raw)
To: Guillaume Lelarge <[email protected]>; +Cc: Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Thu, 21 Nov 2024 at 08:30, Guillaume Lelarge <[email protected]> wrote:
> OK, I'm fine with this. v4 patch attached with one plan showing read, written, and dirtied buffers.
I think this might be a good time for anyone out there who is against
turning on BUFFERS when ANALYZE is on to speak up.
Votes for changing this so far seem to be: Me, Michael Christofides,
Guillaume Lelarge, Robert, Greg Sabino Mullane, and David Fetter (from
2020) [1].
Votes against: None
David
[1] https://www.postgresql.org/message-id/20200601010025.GS13741%40fetter.org
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-11-20 21:35 Bruce Momjian <[email protected]>
parent: David Rowley <[email protected]>
3 siblings, 0 replies; 32+ messages in thread
From: Bruce Momjian @ 2024-11-20 21:35 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Guillaume Lelarge <[email protected]>; Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Thu, Nov 21, 2024 at 10:22:54AM +1300, David Rowley wrote:
> On Thu, 21 Nov 2024 at 08:30, Guillaume Lelarge <[email protected]> wrote:
> > OK, I'm fine with this. v4 patch attached with one plan showing read, written, and dirtied buffers.
>
> I think this might be a good time for anyone out there who is against
> turning on BUFFERS when ANALYZE is on to speak up.
>
> Votes for changing this so far seem to be: Me, Michael Christofides,
> Guillaume Lelarge, Robert, Greg Sabino Mullane, and David Fetter (from
> 2020) [1].
Please add me to the above list. What we have now is too confusing.
--
Bruce Momjian <[email protected]> https://momjian.us
EDB https://enterprisedb.com
When a patient asks the doctor, "Am I going to die?", he means
"Am I going to die soon?"
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-11-20 21:54 Vik Fearing <[email protected]>
parent: David Rowley <[email protected]>
3 siblings, 1 reply; 32+ messages in thread
From: Vik Fearing @ 2024-11-20 21:54 UTC (permalink / raw)
To: David Rowley <[email protected]>; Guillaume Lelarge <[email protected]>; +Cc: Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On 20/11/2024 22:22, David Rowley wrote:
> On Thu, 21 Nov 2024 at 08:30, Guillaume Lelarge <[email protected]> wrote:
>> OK, I'm fine with this. v4 patch attached with one plan showing read, written, and dirtied buffers.
> I think this might be a good time for anyone out there who is against
> turning on BUFFERS when ANALYZE is on to speak up.
>
> Votes for changing this so far seem to be: Me, Michael Christofides,
> Guillaume Lelarge, Robert, Greg Sabino Mullane, and David Fetter (from
> 2020) [1].
Add me to the pro list, please.
https://www.postgresql.org/message-id/[email protected]
--
Vik Fearing
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-11-21 05:13 Laurenz Albe <[email protected]>
parent: Vik Fearing <[email protected]>
0 siblings, 1 reply; 32+ messages in thread
From: Laurenz Albe @ 2024-11-21 05:13 UTC (permalink / raw)
To: Vik Fearing <[email protected]>; David Rowley <[email protected]>; Guillaume Lelarge <[email protected]>; +Cc: Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Wed, 2024-11-20 at 22:54 +0100, Vik Fearing wrote:
> On 20/11/2024 22:22, David Rowley wrote:
> > On Thu, 21 Nov 2024 at 08:30, Guillaume Lelarge <[email protected]> wrote:
> > > OK, I'm fine with this. v4 patch attached with one plan showing read, written, and dirtied buffers.
> > I think this might be a good time for anyone out there who is against
> > turning on BUFFERS when ANALYZE is on to speak up.
> >
> > Votes for changing this so far seem to be: Me, Michael Christofides,
> > Guillaume Lelarge, Robert, Greg Sabino Mullane, and David Fetter (from
> > 2020) [1].
>
> Add me to the pro list, please.
>
> https://www.postgresql.org/message-id/[email protected]
Me, too!
https://postgr.es/m/[email protected]
Yours,
Laurenz Albe
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-11-25 20:44 Robert Haas <[email protected]>
parent: David Rowley <[email protected]>
3 siblings, 1 reply; 32+ messages in thread
From: Robert Haas @ 2024-11-25 20:44 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Guillaume Lelarge <[email protected]>; Greg Sabino Mullane <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Wed, Nov 20, 2024 at 4:23 PM David Rowley <[email protected]> wrote:
> On Thu, 21 Nov 2024 at 08:30, Guillaume Lelarge <[email protected]> wrote:
> > OK, I'm fine with this. v4 patch attached with one plan showing read, written, and dirtied buffers.
>
> I think this might be a good time for anyone out there who is against
> turning on BUFFERS when ANALYZE is on to speak up.
>
> Votes for changing this so far seem to be: Me, Michael Christofides,
> Guillaume Lelarge, Robert, Greg Sabino Mullane, and David Fetter (from
> 2020) [1].
>
> Votes against: None
Just to be clear, my vote is more like +0 than +1. I recognize that
changing this is popular and I'm not opposed to it, but I'm also not
unhappy with things as they are.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-11-25 22:52 David Rowley <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 2 replies; 32+ messages in thread
From: David Rowley @ 2024-11-25 22:52 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Guillaume Lelarge <[email protected]>; Greg Sabino Mullane <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Tue, 26 Nov 2024 at 09:44, Robert Haas <[email protected]> wrote:
>
> On Wed, Nov 20, 2024 at 4:23 PM David Rowley <[email protected]> wrote:
> > I think this might be a good time for anyone out there who is against
> > turning on BUFFERS when ANALYZE is on to speak up.
> >
> > Votes for changing this so far seem to be: Me, Michael Christofides,
> > Guillaume Lelarge, Robert, Greg Sabino Mullane, and David Fetter (from
> > 2020) [1].
> >
> > Votes against: None
>
> Just to be clear, my vote is more like +0 than +1. I recognize that
> changing this is popular and I'm not opposed to it, but I'm also not
> unhappy with things as they are.
OK, thanks. I likely took your neutrality here in a more positive
light after reading the 4-year-old message at [1]. That probably
wasn't the right thing to do.
There are still no votes against it, so unless some come in, I plan to
start looking at the patches proposed to turn buffers on with analyze
with my committer hat on.
David
[1] https://www.postgresql.org/message-id/CA%2BTgmoYH_p-y%3D45SAJ58cU6jsMH6ojgqQZiA2aePpvZ0J%2BuLbA%40ma...
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-11-25 23:12 Tom Lane <[email protected]>
parent: David Rowley <[email protected]>
1 sibling, 1 reply; 32+ messages in thread
From: Tom Lane @ 2024-11-25 23:12 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Robert Haas <[email protected]>; Guillaume Lelarge <[email protected]>; Greg Sabino Mullane <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; pgsql-hackers
David Rowley <[email protected]> writes:
> There are still no votes against it, so unless some come in, I plan to
> start looking at the patches proposed to turn buffers on with analyze
> with my committer hat on.
I'm kind of -0.5, but I'd not bothered to vote because it was
pretty clear what the result was going to be.
regards, tom lane
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-11-27 03:02 David Rowley <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 0 replies; 32+ messages in thread
From: David Rowley @ 2024-11-27 03:02 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Robert Haas <[email protected]>; Guillaume Lelarge <[email protected]>; Greg Sabino Mullane <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; pgsql-hackers
On Tue, 26 Nov 2024 at 12:12, Tom Lane <[email protected]> wrote:
> I'm kind of -0.5, but I'd not bothered to vote because it was
> pretty clear what the result was going to be.
Just so I can properly understand, how much of the -0.5 is "don't like
the idea" vs "rather make changes to well-known behaviour"?
David
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-11-27 09:16 Daniel Gustafsson <[email protected]>
parent: David Rowley <[email protected]>
3 siblings, 0 replies; 32+ messages in thread
From: Daniel Gustafsson @ 2024-11-27 09:16 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Guillaume Lelarge <[email protected]>; Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
> On 20 Nov 2024, at 22:22, David Rowley <[email protected]> wrote:
>
> On Thu, 21 Nov 2024 at 08:30, Guillaume Lelarge <[email protected]> wrote:
>> OK, I'm fine with this. v4 patch attached with one plan showing read, written, and dirtied buffers.
>
> I think this might be a good time for anyone out there who is against
> turning on BUFFERS when ANALYZE is on to speak up.
>
> Votes for changing this so far seem to be: Me, Michael Christofides,
> Guillaume Lelarge, Robert, Greg Sabino Mullane, and David Fetter (from
> 2020) [1].
I would also like to see this on by default, while it does make it more verbose
the extra data is valuable in debugging. +1 from me.
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-11-27 16:18 Robert Haas <[email protected]>
parent: David Rowley <[email protected]>
1 sibling, 0 replies; 32+ messages in thread
From: Robert Haas @ 2024-11-27 16:18 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Guillaume Lelarge <[email protected]>; Greg Sabino Mullane <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Mon, Nov 25, 2024 at 5:52 PM David Rowley <[email protected]> wrote:
> OK, thanks. I likely took your neutrality here in a more positive
> light after reading the 4-year-old message at [1]. That probably
> wasn't the right thing to do.
Yeah, I think my view has wavered over time. Probably when I wrote
that message I had recently seen some cases where the additional
detail would have been helpful; but as of this writing the cases I've
seen most recently are those where the output is mind-numbingly long
already.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-12-02 11:41 Michael Christofides <[email protected]>
parent: Laurenz Albe <[email protected]>
0 siblings, 1 reply; 32+ messages in thread
From: Michael Christofides @ 2024-12-02 11:41 UTC (permalink / raw)
To: Laurenz Albe <[email protected]>; +Cc: Vik Fearing <[email protected]>; David Rowley <[email protected]>; Guillaume Lelarge <[email protected]>; Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
> v4 patch attached
Thank you Guillaume, and nice work! I tried to see if there was anywhere
else in the documentation that would need updating, but it looks like you
covered everywhere already.
> I'm with Robert in that I've not found the buffer counts to be all that
useful most of the time.
I find the buffer counts especially helpful for educating newer folks on
why things are slow, even when they are not necessary for spotting the
issue (for more advanced users). One of my hopes is that by educating and
empowering newer users on how I/O relates to performance issues, fewer
cases will get escalated to more experienced folks.
> the cases I've seen most recently are those where the output is
mind-numbingly long already.
Are you mostly seeing query plans that have stumped other people already
(eg second or third line support), so perhaps seeing more complex plans
than the average user?
Both Depesz[1] and Tensor[2] have archives of publicly submitted plans,
which I found helpful for checking how slow plans look for users of those
tools. I have a similar archive, and while we do not publish them (and
there are plenty of huge plans) it also suggests that the majority of slow
plans people are reviewing have fewer than 20 nodes.
I realise it’s optimistic to think that the time experienced hackers would
lose having to sift through longer plans would be gained back by having to
do so less often, but I thought it was worth raising as part of the aim.
I also looked into the Slow Query Questions page on the wiki that we ask
people to review before posting to pgsql-performance, and noticed that has
suggested requesting buffers for the past 12 years[3].
—
Michael
[1]: https://explain.depesz.com/history
[2]: https://explain.tensor.ru/archive
[3]:
https://wiki.postgresql.org/index.php?title=Slow_Query_Questions&diff=18308&oldid=16800
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-12-03 14:32 Robert Haas <[email protected]>
parent: Michael Christofides <[email protected]>
0 siblings, 0 replies; 32+ messages in thread
From: Robert Haas @ 2024-12-03 14:32 UTC (permalink / raw)
To: Michael Christofides <[email protected]>; +Cc: Laurenz Albe <[email protected]>; Vik Fearing <[email protected]>; David Rowley <[email protected]>; Guillaume Lelarge <[email protected]>; Greg Sabino Mullane <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Mon, Dec 2, 2024 at 6:42 AM Michael Christofides
<[email protected]> wrote:
> Are you mostly seeing query plans that have stumped other people already (eg second or third line support), so perhaps seeing more complex plans than the average user?
Yes. :-)
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-12-10 02:57 David Rowley <[email protected]>
parent: Guillaume Lelarge <[email protected]>
1 sibling, 1 reply; 32+ messages in thread
From: David Rowley @ 2024-12-10 02:57 UTC (permalink / raw)
To: Guillaume Lelarge <[email protected]>; +Cc: Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Thu, 21 Nov 2024 at 08:30, Guillaume Lelarge <[email protected]> wrote:
> OK, I'm fine with this. v4 patch attached with one plan showing read, written, and dirtied buffers.
Today I spent more time polishing this patch. There were a few cases
in the docs that displayed EXPLAIN ANALYZE output that you'd not
adjusted to include the buffers output or adjusted to do BUFFERS OFF.
I think I've got all these now. Tom went to some effort to fix some
outdated EXPLAIN outputs for v17 in 5caa05749, so I think we owe it to
him not to let these go out of date so soon after that change.
I also was thinking again about what Robert mentioned about
auto_explain.log_buffers should now also be on by default. I'm less
certain than him about this change. It seems like a separate
consideration that we could apply many of the same arguments for the
main change to. In any case, I extracted that change from the 0001
patch and put it in a 0002 patch as it doesn't seem like something
that should be a sidenote in the commit message. I felt doing that
increases the chances that it would be overlooked in the release
notes.
I was very close to pushing 0001 today, but will hold off until
tomorrow to see if anyone has final comments.
For 0002, I'd really like to see a bit more justification for it. For
the record, I'm not against 0002, it's just that my personal arguments
for wanting 0001 don't apply to 0002.
David
Attachments:
[application/octet-stream] v5-0001-Enable-BUFFERS-by-default-with-EXPLAIN-ANALYZE.patch (97.7K, ../../CAApHDvpj0Cj6Z8VnDN5htzF24b4_hKL+1_n=HOehLHkBK_E_TQ@mail.gmail.com/2-v5-0001-Enable-BUFFERS-by-default-with-EXPLAIN-ANALYZE.patch)
download | inline diff:
From 8dc51d23c0c4d2bd8b9e73db4cc7d82a86439baf Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 10 Dec 2024 12:50:20 +1300
Subject: [PATCH v5 1/2] Enable BUFFERS by default with EXPLAIN ANALYZE
The topic of turning EXPLAIN's BUFFERS option on with the ANALYZE option
has come up a few times over the past few years. In many ways, doing this
seems like a good idea as it may be more obvious to users why a given
query is running more slowly than they might expect. Also, from my own
(David's) personal experience, I've seen users posting to the mailing
lists with two identical plans, one slow and one fast asking why their
query is sometimes slow. In many cases, this is due to additional reads.
Having BUFFERS on by default may help reduce some of these questions, and
if not, make it more obvious to the user before they post, or save a
round-trip to the mailing list when additional I/O effort is the cause of
the slowness.
The general consensus is that we want BUFFERS on by default with
ANALYZE, but there were more than zero concerns raised with doing so.
The primary reason against is the additional verbosity, making it harder
to read large plans. Another concern was that buffer information isn't
always useful so may not make sense to have it on by default. There
appeared to be very little to no concern from simply breaking the status
quo.
It's currently December, so let's commit this to see if any more people
come forward to argue against this change. We have over half a year
remaining in the v18 cycle where we still could consider reverting this if
enough people come forward with objections or if another reason against
doing this materializes.
There were two patches independently submitted to achieve this goal, one
by me and the other by Guillaume. This commit is a mix of both of these
patches with some additional work done by me to adjust various
additional places in the documentation which include EXPLAIN ANALYZE
output.
Author: Guillaume Lelarge, David Rowley
Reviewed-by: Robert Haas, Greg Sabino Mullane, Michael Christofides
Discussion: https://postgr.es/m/CANNMO++W7MM8T0KyXN3ZheXXt-uLVM3aEtZd+WNfZ=obxffUiA@mail.gmail.com
---
.../postgres_fdw/expected/postgres_fdw.out | 8 +-
contrib/postgres_fdw/sql/postgres_fdw.sql | 8 +-
doc/src/sgml/bloom.sgml | 16 +++-
doc/src/sgml/jit.sgml | 5 +-
doc/src/sgml/perform.sgml | 44 ++++++---
doc/src/sgml/planstats.sgml | 18 ++--
doc/src/sgml/ref/explain.sgml | 10 +-
src/backend/commands/explain.c | 7 ++
src/test/regress/expected/brin_multi.out | 18 ++--
src/test/regress/expected/explain.out | 38 ++++++--
.../regress/expected/incremental_sort.out | 4 +-
src/test/regress/expected/memoize.out | 2 +-
src/test/regress/expected/merge.out | 2 +-
src/test/regress/expected/partition_prune.out | 96 +++++++++----------
src/test/regress/expected/select.out | 2 +-
src/test/regress/expected/select_into.out | 4 +-
src/test/regress/expected/select_parallel.out | 6 +-
src/test/regress/expected/subselect.out | 2 +-
src/test/regress/expected/tidscan.out | 6 +-
src/test/regress/sql/brin_multi.sql | 18 ++--
src/test/regress/sql/explain.sql | 16 ++--
src/test/regress/sql/incremental_sort.sql | 4 +-
src/test/regress/sql/memoize.sql | 2 +-
src/test/regress/sql/merge.sql | 2 +-
src/test/regress/sql/partition_prune.sql | 96 +++++++++----------
src/test/regress/sql/select.sql | 2 +-
src/test/regress/sql/select_into.sql | 4 +-
src/test/regress/sql/select_parallel.sql | 6 +-
src/test/regress/sql/subselect.sql | 2 +-
src/test/regress/sql/tidscan.sql | 6 +-
30 files changed, 257 insertions(+), 197 deletions(-)
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index f2bcd6aa98..bf322198a2 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -11553,7 +11553,7 @@ SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c
Filter: (async_pt_3.a = local_tbl.a)
(15 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
QUERY PLAN
-------------------------------------------------------------------------------
@@ -11799,7 +11799,7 @@ SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt W
Remote SQL: SELECT a, b, c FROM public.base_tbl2 WHERE ((a < 3000))
(20 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
QUERY PLAN
-----------------------------------------------------------------------------------------
@@ -11843,7 +11843,7 @@ SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
Filter: (t1_3.b === 505)
(14 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
QUERY PLAN
-------------------------------------------------------------------------
@@ -12003,7 +12003,7 @@ DELETE FROM async_pt WHERE b = 0 RETURNING *;
DELETE FROM async_p1;
DELETE FROM async_p2;
DELETE FROM async_p3;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
SELECT * FROM async_pt;
QUERY PLAN
-------------------------------------------------------------------------
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 372fe6dad1..3900522ccb 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -3904,7 +3904,7 @@ ALTER FOREIGN TABLE async_p2 OPTIONS (use_remote_estimate 'true');
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
@@ -3979,13 +3979,13 @@ ANALYZE local_tbl;
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
@@ -4037,7 +4037,7 @@ DELETE FROM async_p1;
DELETE FROM async_p2;
DELETE FROM async_p3;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
SELECT * FROM async_pt;
-- Clean up
diff --git a/doc/src/sgml/bloom.sgml b/doc/src/sgml/bloom.sgml
index 0e734f3d6e..6a8a60b8c7 100644
--- a/doc/src/sgml/bloom.sgml
+++ b/doc/src/sgml/bloom.sgml
@@ -121,9 +121,10 @@ SELECT 10000000
Seq Scan on tbloom (cost=0.00..213744.00 rows=250 width=24) (actual time=357.059..357.059 rows=0 loops=1)
Filter: ((i2 = 898732) AND (i5 = 123451))
Rows Removed by Filter: 10000000
+ Buffers: shared hit=63744
Planning Time: 0.346 ms
Execution Time: 357.076 ms
-(5 rows)
+(6 rows)
</programlisting>
</para>
@@ -144,9 +145,10 @@ CREATE INDEX
Seq Scan on tbloom (cost=0.00..213744.00 rows=2 width=24) (actual time=351.016..351.017 rows=0 loops=1)
Filter: ((i2 = 898732) AND (i5 = 123451))
Rows Removed by Filter: 10000000
+ Buffers: shared hit=63744
Planning Time: 0.138 ms
Execution Time: 351.035 ms
-(5 rows)
+(6 rows)
</programlisting>
</para>
@@ -168,11 +170,13 @@ CREATE INDEX
Recheck Cond: ((i2 = 898732) AND (i5 = 123451))
Rows Removed by Index Recheck: 2300
Heap Blocks: exact=2256
+ Buffers: shared hit=21864
-> Bitmap Index Scan on bloomidx (cost=0.00..178436.00 rows=1 width=0) (actual time=20.005..20.005 rows=2300 loops=1)
Index Cond: ((i2 = 898732) AND (i5 = 123451))
+ Buffers: shared hit=19608
Planning Time: 0.099 ms
Execution Time: 22.632 ms
-(8 rows)
+(10 rows)
</programlisting>
</para>
@@ -199,14 +203,18 @@ CREATE INDEX
-------------------------------------------------------------------&zwsp;--------------------------------------------------------
Bitmap Heap Scan on tbloom (cost=9.29..13.30 rows=1 width=24) (actual time=0.032..0.033 rows=0 loops=1)
Recheck Cond: ((i5 = 123451) AND (i2 = 898732))
+ Buffers: shared read=6
-> BitmapAnd (cost=9.29..9.29 rows=1 width=0) (actual time=0.047..0.047 rows=0 loops=1)
+ Buffers: shared hit=6
-> Bitmap Index Scan on btreeidx5 (cost=0.00..4.52 rows=11 width=0) (actual time=0.026..0.026 rows=7 loops=1)
Index Cond: (i5 = 123451)
+ Buffers: shared hit=3
-> Bitmap Index Scan on btreeidx2 (cost=0.00..4.52 rows=11 width=0) (actual time=0.007..0.007 rows=8 loops=1)
Index Cond: (i2 = 898732)
+ Buffers: shared hit=3
Planning Time: 0.264 ms
Execution Time: 0.047 ms
-(9 rows)
+(13 rows)
</programlisting>
Although this query runs much faster than with either of the single
indexes, we pay a penalty in index size. Each of the single-column
diff --git a/doc/src/sgml/jit.sgml b/doc/src/sgml/jit.sgml
index 1921557cb8..458f8acb41 100644
--- a/doc/src/sgml/jit.sgml
+++ b/doc/src/sgml/jit.sgml
@@ -149,10 +149,11 @@
QUERY PLAN
-------------------------------------------------------------------&zwsp;------------------------------------------
Aggregate (cost=16.27..16.29 rows=1 width=8) (actual time=0.303..0.303 rows=1 loops=1)
+ Buffers: shared hit=14
-> Seq Scan on pg_class (cost=0.00..15.42 rows=342 width=4) (actual time=0.017..0.111 rows=356 loops=1)
+ Buffers: shared hit=14
Planning Time: 0.116 ms
Execution Time: 0.365 ms
-(4 rows)
</screen>
Given the cost of the plan, it is entirely reasonable that no
<acronym>JIT</acronym> was used; the cost of <acronym>JIT</acronym> would
@@ -165,7 +166,9 @@ SET
QUERY PLAN
-------------------------------------------------------------------&zwsp;------------------------------------------
Aggregate (cost=16.27..16.29 rows=1 width=8) (actual time=6.049..6.049 rows=1 loops=1)
+ Buffers: shared hit=14
-> Seq Scan on pg_class (cost=0.00..15.42 rows=342 width=4) (actual time=0.019..0.052 rows=356 loops=1)
+ Buffers: shared hit=14
Planning Time: 0.133 ms
JIT:
Functions: 3
diff --git a/doc/src/sgml/perform.sgml b/doc/src/sgml/perform.sgml
index cd12b9ce48..a502a2aaba 100644
--- a/doc/src/sgml/perform.sgml
+++ b/doc/src/sgml/perform.sgml
@@ -39,7 +39,7 @@
<para>
Examples in this section are drawn from the regression test database
- after doing a <command>VACUUM ANALYZE</command>, using v17 development sources.
+ after doing a <command>VACUUM ANALYZE</command>, using v18 development sources.
You should be able to get similar results if you try the examples
yourself, but your estimated costs and row counts might vary slightly
because <command>ANALYZE</command>'s statistics are random samples rather
@@ -722,13 +722,19 @@ WHERE t1.unique1 < 10 AND t1.unique2 = t2.unique2;
QUERY PLAN
-------------------------------------------------------------------&zwsp;--------------------------------------------------------------
Nested Loop (cost=4.65..118.50 rows=10 width=488) (actual time=0.017..0.051 rows=10 loops=1)
+ Buffers: shared hit=36 read=6
-> Bitmap Heap Scan on tenk1 t1 (cost=4.36..39.38 rows=10 width=244) (actual time=0.009..0.017 rows=10 loops=1)
Recheck Cond: (unique1 < 10)
Heap Blocks: exact=10
+ Buffers: shared hit=3 read=5 written=4
-> Bitmap Index Scan on tenk1_unique1 (cost=0.00..4.36 rows=10 width=0) (actual time=0.004..0.004 rows=10 loops=1)
Index Cond: (unique1 < 10)
+ Buffers: shared hit=2
-> Index Scan using tenk2_unique2 on tenk2 t2 (cost=0.29..7.90 rows=1 width=244) (actual time=0.003..0.003 rows=1 loops=10)
Index Cond: (unique2 = t1.unique2)
+ Buffers: shared hit=24 read=6
+ Planning:
+ Buffers: shared hit=15 dirtied=9
Planning Time: 0.485 ms
Execution Time: 0.073 ms
</screen>
@@ -769,16 +775,24 @@ WHERE t1.unique1 < 100 AND t1.unique2 = t2.unique2 ORDER BY t1.fivethous;
Sort (cost=713.05..713.30 rows=100 width=488) (actual time=2.995..3.002 rows=100 loops=1)
Sort Key: t1.fivethous
Sort Method: quicksort Memory: 74kB
+ Buffers: shared hit=440
-> Hash Join (cost=226.23..709.73 rows=100 width=488) (actual time=0.515..2.920 rows=100 loops=1)
Hash Cond: (t2.unique2 = t1.unique2)
+ Buffers: shared hit=437
-> Seq Scan on tenk2 t2 (cost=0.00..445.00 rows=10000 width=244) (actual time=0.026..1.790 rows=10000 loops=1)
+ Buffers: shared hit=345
-> Hash (cost=224.98..224.98 rows=100 width=244) (actual time=0.476..0.477 rows=100 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 35kB
+ Buffers: shared hit=92
-> Bitmap Heap Scan on tenk1 t1 (cost=5.06..224.98 rows=100 width=244) (actual time=0.030..0.450 rows=100 loops=1)
Recheck Cond: (unique1 < 100)
Heap Blocks: exact=90
+ Buffers: shared hit=92
-> Bitmap Index Scan on tenk1_unique1 (cost=0.00..5.04 rows=100 width=0) (actual time=0.013..0.013 rows=100 loops=1)
Index Cond: (unique1 < 100)
+ Buffers: shared hit=2
+ Planning:
+ Buffers: shared hit=12
Planning Time: 0.187 ms
Execution Time: 3.036 ms
</screen>
@@ -803,6 +817,7 @@ EXPLAIN ANALYZE SELECT * FROM tenk1 WHERE ten < 7;
Seq Scan on tenk1 (cost=0.00..470.00 rows=7000 width=244) (actual time=0.030..1.995 rows=7000 loops=1)
Filter: (ten < 7)
Rows Removed by Filter: 3000
+ Buffers: shared hit=345
Planning Time: 0.102 ms
Execution Time: 2.145 ms
</screen>
@@ -826,6 +841,7 @@ EXPLAIN ANALYZE SELECT * FROM polygon_tbl WHERE f1 @> polygon '(0.5,2.0)';
Seq Scan on polygon_tbl (cost=0.00..1.09 rows=1 width=85) (actual time=0.023..0.023 rows=0 loops=1)
Filter: (f1 @> '((0.5,2))'::polygon)
Rows Removed by Filter: 7
+ Buffers: shared hit=1
Planning Time: 0.039 ms
Execution Time: 0.033 ms
</screen>
@@ -845,6 +861,7 @@ EXPLAIN ANALYZE SELECT * FROM polygon_tbl WHERE f1 @> polygon '(0.5,2.0)';
Index Scan using gpolygonind on polygon_tbl (cost=0.13..8.15 rows=1 width=85) (actual time=0.074..0.074 rows=0 loops=1)
Index Cond: (f1 @> '((0.5,2))'::polygon)
Rows Removed by Index Recheck: 1
+ Buffers: shared hit=1
Planning Time: 0.039 ms
Execution Time: 0.098 ms
</screen>
@@ -857,34 +874,31 @@ EXPLAIN ANALYZE SELECT * FROM polygon_tbl WHERE f1 @> polygon '(0.5,2.0)';
</para>
<para>
- <command>EXPLAIN</command> has a <literal>BUFFERS</literal> option that can be used with
- <literal>ANALYZE</literal> to get even more run time statistics:
+ <command>EXPLAIN</command> has a <literal>BUFFERS</literal> option which
+ provides additional detail about I/O operations performed during the
+ planning and execution of the given query. The buffer numbers displayed
+ show the count of the non-distinct buffers hit, read, dirtied, and written
+ for the given node and all of its child nodes. The
+ <literal>ANALYZE</literal> option implicitly enables the
+ <literal>BUFFERS</literal> option. If this
+ is undesired, <literal>BUFFERS</literal> may be explicitly disabled:
<screen>
-EXPLAIN (ANALYZE, BUFFERS) SELECT * FROM tenk1 WHERE unique1 < 100 AND unique2 > 9000;
+EXPLAIN (ANALYZE, BUFFERS OFF) SELECT * FROM tenk1 WHERE unique1 < 100 AND unique2 > 9000;
QUERY PLAN
-------------------------------------------------------------------&zwsp;--------------------------------------------------------------
Bitmap Heap Scan on tenk1 (cost=25.07..60.11 rows=10 width=244) (actual time=0.105..0.114 rows=10 loops=1)
Recheck Cond: ((unique1 < 100) AND (unique2 > 9000))
Heap Blocks: exact=10
- Buffers: shared hit=14 read=3
-> BitmapAnd (cost=25.07..25.07 rows=10 width=0) (actual time=0.100..0.101 rows=0 loops=1)
- Buffers: shared hit=4 read=3
-> Bitmap Index Scan on tenk1_unique1 (cost=0.00..5.04 rows=100 width=0) (actual time=0.027..0.027 rows=100 loops=1)
Index Cond: (unique1 < 100)
- Buffers: shared hit=2
-> Bitmap Index Scan on tenk1_unique2 (cost=0.00..19.78 rows=999 width=0) (actual time=0.070..0.070 rows=999 loops=1)
Index Cond: (unique2 > 9000)
- Buffers: shared hit=2 read=3
- Planning:
- Buffers: shared hit=3
Planning Time: 0.162 ms
Execution Time: 0.143 ms
</screen>
-
- The numbers provided by <literal>BUFFERS</literal> help to identify which parts
- of the query are the most I/O-intensive.
</para>
<para>
@@ -906,8 +920,10 @@ EXPLAIN ANALYZE UPDATE tenk1 SET hundred = hundred + 1 WHERE unique1 < 100;
-> Bitmap Heap Scan on tenk1 (cost=5.06..225.23 rows=100 width=10) (actual time=0.065..0.141 rows=100 loops=1)
Recheck Cond: (unique1 < 100)
Heap Blocks: exact=90
+ Buffers: shared hit=4 read=2
-> Bitmap Index Scan on tenk1_unique1 (cost=0.00..5.04 rows=100 width=0) (actual time=0.031..0.031 rows=100 loops=1)
Index Cond: (unique1 < 100)
+ Buffers: shared read=2
Planning Time: 0.151 ms
Execution Time: 1.856 ms
@@ -1040,10 +1056,12 @@ EXPLAIN ANALYZE SELECT * FROM tenk1 WHERE unique1 < 100 AND unique2 > 9000
QUERY PLAN
-------------------------------------------------------------------&zwsp;------------------------------------------------------------
Limit (cost=0.29..14.33 rows=2 width=244) (actual time=0.051..0.071 rows=2 loops=1)
+ Buffers: shared hit=16
-> Index Scan using tenk1_unique2 on tenk1 (cost=0.29..70.50 rows=10 width=244) (actual time=0.051..0.070 rows=2 loops=1)
Index Cond: (unique2 > 9000)
Filter: (unique1 < 100)
Rows Removed by Filter: 287
+ Buffers: shared hit=16
Planning Time: 0.077 ms
Execution Time: 0.086 ms
</screen>
diff --git a/doc/src/sgml/planstats.sgml b/doc/src/sgml/planstats.sgml
index c7ec749d0a..c957f4f362 100644
--- a/doc/src/sgml/planstats.sgml
+++ b/doc/src/sgml/planstats.sgml
@@ -489,7 +489,7 @@ SELECT relpages, reltuples FROM pg_class WHERE relname = 't';
condition on the <structfield>a</structfield> column:
<programlisting>
-EXPLAIN (ANALYZE, TIMING OFF) SELECT * FROM t WHERE a = 1;
+EXPLAIN (ANALYZE, TIMING OFF, BUFFERS OFF) SELECT * FROM t WHERE a = 1;
QUERY PLAN
-------------------------------------------------------------------&zwsp;------------
Seq Scan on t (cost=0.00..170.00 rows=100 width=8) (actual rows=100 loops=1)
@@ -506,7 +506,7 @@ EXPLAIN (ANALYZE, TIMING OFF) SELECT * FROM t WHERE a = 1;
condition on both columns, combining them with <literal>AND</literal>:
<programlisting>
-EXPLAIN (ANALYZE, TIMING OFF) SELECT * FROM t WHERE a = 1 AND b = 1;
+EXPLAIN (ANALYZE, TIMING OFF, BUFFERS OFF) SELECT * FROM t WHERE a = 1 AND b = 1;
QUERY PLAN
-------------------------------------------------------------------&zwsp;----------
Seq Scan on t (cost=0.00..195.00 rows=1 width=8) (actual rows=100 loops=1)
@@ -530,7 +530,7 @@ EXPLAIN (ANALYZE, TIMING OFF) SELECT * FROM t WHERE a = 1 AND b = 1;
<programlisting>
CREATE STATISTICS stts (dependencies) ON a, b FROM t;
ANALYZE t;
-EXPLAIN (ANALYZE, TIMING OFF) SELECT * FROM t WHERE a = 1 AND b = 1;
+EXPLAIN (ANALYZE, TIMING OFF, BUFFERS OFF) SELECT * FROM t WHERE a = 1 AND b = 1;
QUERY PLAN
-------------------------------------------------------------------&zwsp;------------
Seq Scan on t (cost=0.00..195.00 rows=100 width=8) (actual rows=100 loops=1)
@@ -551,7 +551,7 @@ EXPLAIN (ANALYZE, TIMING OFF) SELECT * FROM t WHERE a = 1 AND b = 1;
estimated number of rows returned by the HashAggregate node) is very
accurate:
<programlisting>
-EXPLAIN (ANALYZE, TIMING OFF) SELECT COUNT(*) FROM t GROUP BY a;
+EXPLAIN (ANALYZE, TIMING OFF, BUFFERS OFF) SELECT COUNT(*) FROM t GROUP BY a;
QUERY PLAN
-------------------------------------------------------------------&zwsp;----------------------
HashAggregate (cost=195.00..196.00 rows=100 width=12) (actual rows=100 loops=1)
@@ -562,7 +562,7 @@ EXPLAIN (ANALYZE, TIMING OFF) SELECT COUNT(*) FROM t GROUP BY a;
groups in a query with two columns in <command>GROUP BY</command>, as
in the following example, is off by an order of magnitude:
<programlisting>
-EXPLAIN (ANALYZE, TIMING OFF) SELECT COUNT(*) FROM t GROUP BY a, b;
+EXPLAIN (ANALYZE, TIMING OFF, BUFFERS OFF) SELECT COUNT(*) FROM t GROUP BY a, b;
QUERY PLAN
-------------------------------------------------------------------&zwsp;-------------------------
HashAggregate (cost=220.00..230.00 rows=1000 width=16) (actual rows=100 loops=1)
@@ -575,7 +575,7 @@ EXPLAIN (ANALYZE, TIMING OFF) SELECT COUNT(*) FROM t GROUP BY a, b;
DROP STATISTICS stts;
CREATE STATISTICS stts (dependencies, ndistinct) ON a, b FROM t;
ANALYZE t;
-EXPLAIN (ANALYZE, TIMING OFF) SELECT COUNT(*) FROM t GROUP BY a, b;
+EXPLAIN (ANALYZE, TIMING OFF, BUFFERS OFF) SELECT COUNT(*) FROM t GROUP BY a, b;
QUERY PLAN
-------------------------------------------------------------------&zwsp;-------------------------
HashAggregate (cost=220.00..221.00 rows=100 width=16) (actual rows=100 loops=1)
@@ -615,7 +615,7 @@ EXPLAIN (ANALYZE, TIMING OFF) SELECT COUNT(*) FROM t GROUP BY a, b;
DROP STATISTICS stts;
CREATE STATISTICS stts2 (mcv) ON a, b FROM t;
ANALYZE t;
-EXPLAIN (ANALYZE, TIMING OFF) SELECT * FROM t WHERE a = 1 AND b = 1;
+EXPLAIN (ANALYZE, TIMING OFF, BUFFERS OFF) SELECT * FROM t WHERE a = 1 AND b = 1;
QUERY PLAN
-------------------------------------------------------------------&zwsp;------------
Seq Scan on t (cost=0.00..195.00 rows=100 width=8) (actual rows=100 loops=1)
@@ -672,7 +672,7 @@ SELECT m.* FROM pg_statistic_ext join pg_statistic_ext_data on (oid = stxoid),
to decide which combinations are compatible.
<programlisting>
-EXPLAIN (ANALYZE, TIMING OFF) SELECT * FROM t WHERE a = 1 AND b = 10;
+EXPLAIN (ANALYZE, TIMING OFF, BUFFERS OFF) SELECT * FROM t WHERE a = 1 AND b = 10;
QUERY PLAN
-------------------------------------------------------------------&zwsp;--------
Seq Scan on t (cost=0.00..195.00 rows=1 width=8) (actual rows=0 loops=1)
@@ -685,7 +685,7 @@ EXPLAIN (ANALYZE, TIMING OFF) SELECT * FROM t WHERE a = 1 AND b = 10;
consider the following range query for the same table:
<programlisting>
-EXPLAIN (ANALYZE, TIMING OFF) SELECT * FROM t WHERE a <= 49 AND b > 49;
+EXPLAIN (ANALYZE, TIMING OFF, BUFFERS OFF) SELECT * FROM t WHERE a <= 49 AND b > 49;
QUERY PLAN
-------------------------------------------------------------------&zwsp;--------
Seq Scan on t (cost=0.00..195.00 rows=1 width=8) (actual rows=0 loops=1)
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index db9d3a8549..6361a14e65 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -201,8 +201,10 @@ ROLLBACK;
query processing.
The number of blocks shown for an
upper-level node includes those used by all its child nodes. In text
- format, only non-zero values are printed. This parameter defaults to
- <literal>FALSE</literal>.
+ format, only non-zero values are printed. Buffers information is
+ included by default when <literal>ANALYZE</literal> is used but
+ otherwise is not included by default, but can be enabled using this
+ option.
</para>
</listitem>
</varlistentry>
@@ -500,11 +502,13 @@ EXPLAIN ANALYZE EXECUTE query(100, 200);
HashAggregate (cost=10.77..10.87 rows=10 width=12) (actual time=0.043..0.044 rows=10 loops=1)
Group Key: foo
Batches: 1 Memory Usage: 24kB
+ Buffers: shared hit=4
-> Index Scan using test_pkey on test (cost=0.29..10.27 rows=99 width=8) (actual time=0.009..0.025 rows=99 loops=1)
Index Cond: ((id > 100) AND (id < 200))
+ Buffers: shared hit=4
Planning Time: 0.244 ms
Execution Time: 0.073 ms
-(7 rows)
+(9 rows)
</programlisting>
</para>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 3078f5c1a3..a201ed3082 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -198,6 +198,7 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
List *rewritten;
ListCell *lc;
bool timing_set = false;
+ bool buffers_set = false;
bool summary_set = false;
/* Parse options list. */
@@ -212,7 +213,10 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
else if (strcmp(opt->defname, "costs") == 0)
es->costs = defGetBoolean(opt);
else if (strcmp(opt->defname, "buffers") == 0)
+ {
+ buffers_set = true;
es->buffers = defGetBoolean(opt);
+ }
else if (strcmp(opt->defname, "wal") == 0)
es->wal = defGetBoolean(opt);
else if (strcmp(opt->defname, "settings") == 0)
@@ -292,6 +296,9 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
/* if the timing was not set explicitly, set default value */
es->timing = (timing_set) ? es->timing : es->analyze;
+ /* if the buffers was not set explicitly, set default value */
+ es->buffers = (buffers_set) ? es->buffers : es->analyze;
+
/* check that timing is used with EXPLAIN ANALYZE */
if (es->timing && !es->analyze)
ereport(ERROR,
diff --git a/src/test/regress/expected/brin_multi.out b/src/test/regress/expected/brin_multi.out
index ae9ce9d8ec..f2d1465818 100644
--- a/src/test/regress/expected/brin_multi.out
+++ b/src/test/regress/expected/brin_multi.out
@@ -845,7 +845,7 @@ INSERT INTO brin_date_test SELECT '5874897-12-01'::date + i FROM generate_series
CREATE INDEX ON brin_date_test USING brin (a date_minmax_multi_ops) WITH (pages_per_range=1);
SET enable_seqscan = off;
-- make sure the ranges were built correctly and 2023-01-01 eliminates all
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_date_test WHERE a = '2023-01-01'::date;
QUERY PLAN
-------------------------------------------------------------------------
@@ -864,7 +864,7 @@ INSERT INTO brin_timestamp_test
SELECT i FROM generate_series('2000-01-01'::timestamp, '2000-02-09'::timestamp, '1 day'::interval) s(i);
CREATE INDEX ON brin_timestamp_test USING brin (a timestamp_minmax_multi_ops) WITH (pages_per_range=1);
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_timestamp_test WHERE a = '2023-01-01'::timestamp;
QUERY PLAN
------------------------------------------------------------------------------
@@ -874,7 +874,7 @@ SELECT * FROM brin_timestamp_test WHERE a = '2023-01-01'::timestamp;
Index Cond: (a = '2023-01-01 00:00:00'::timestamp without time zone)
(4 rows)
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_timestamp_test WHERE a = '1900-01-01'::timestamp;
QUERY PLAN
------------------------------------------------------------------------------
@@ -892,7 +892,7 @@ INSERT INTO brin_date_test VALUES ('-infinity'), ('infinity');
INSERT INTO brin_date_test SELECT '2000-01-01'::date + i FROM generate_series(1, 40) s(i);
CREATE INDEX ON brin_date_test USING brin (a date_minmax_multi_ops) WITH (pages_per_range=1);
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_date_test WHERE a = '2023-01-01'::date;
QUERY PLAN
-------------------------------------------------------------------------
@@ -902,7 +902,7 @@ SELECT * FROM brin_date_test WHERE a = '2023-01-01'::date;
Index Cond: (a = '2023-01-01'::date)
(4 rows)
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_date_test WHERE a = '1900-01-01'::date;
QUERY PLAN
-------------------------------------------------------------------------
@@ -921,7 +921,7 @@ INSERT INTO brin_interval_test SELECT (i || ' years')::interval FROM generate_se
INSERT INTO brin_interval_test SELECT (i || ' years')::interval FROM generate_series( 177999980, 178000000) s(i);
CREATE INDEX ON brin_interval_test USING brin (a interval_minmax_multi_ops) WITH (pages_per_range=1);
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '-30 years'::interval;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -931,7 +931,7 @@ SELECT * FROM brin_interval_test WHERE a = '-30 years'::interval;
Index Cond: (a = '@ 30 years ago'::interval)
(4 rows)
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '30 years'::interval;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -949,7 +949,7 @@ INSERT INTO brin_interval_test VALUES ('-infinity'), ('infinity');
INSERT INTO brin_interval_test SELECT (i || ' days')::interval FROM generate_series(100, 140) s(i);
CREATE INDEX ON brin_interval_test USING brin (a interval_minmax_multi_ops) WITH (pages_per_range=1);
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '-30 years'::interval;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -959,7 +959,7 @@ SELECT * FROM brin_interval_test WHERE a = '-30 years'::interval;
Index Cond: (a = '@ 30 years ago'::interval)
(4 rows)
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '30 years'::interval;
QUERY PLAN
-----------------------------------------------------------------------------
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index d2eef8097c..ee31e41d50 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -60,7 +60,7 @@ select explain_filter('explain select * from int8_tbl i8');
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N)
(1 row)
-select explain_filter('explain (analyze) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
explain_filter
-----------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N loops=N)
@@ -68,7 +68,7 @@ select explain_filter('explain (analyze) select * from int8_tbl i8');
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, verbose) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
explain_filter
------------------------------------------------------------------------------------------------------
Seq Scan on public.int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N loops=N)
@@ -352,7 +352,7 @@ select explain_filter('explain (memory) select * from int8_tbl i8');
Memory: used=NkB allocated=NkB
(2 rows)
-select explain_filter('explain (memory, analyze) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, buffers off) select * from int8_tbl i8');
explain_filter
-----------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N loops=N)
@@ -400,9 +400,29 @@ select explain_filter('explain (memory, analyze, format json) select * from int8
"Actual Total Time": N.N, +
"Actual Rows": N, +
"Actual Loops": N, +
- "Disabled": false +
+ "Disabled": false, +
+ "Shared Hit Blocks": N, +
+ "Shared Read Blocks": N, +
+ "Shared Dirtied Blocks": N, +
+ "Shared Written Blocks": N, +
+ "Local Hit Blocks": N, +
+ "Local Read Blocks": N, +
+ "Local Dirtied Blocks": N, +
+ "Local Written Blocks": N, +
+ "Temp Read Blocks": N, +
+ "Temp Written Blocks": N +
}, +
"Planning": { +
+ "Shared Hit Blocks": N, +
+ "Shared Read Blocks": N, +
+ "Shared Dirtied Blocks": N, +
+ "Shared Written Blocks": N, +
+ "Local Hit Blocks": N, +
+ "Local Read Blocks": N, +
+ "Local Dirtied Blocks": N, +
+ "Local Written Blocks": N, +
+ "Temp Read Blocks": N, +
+ "Temp Written Blocks": N, +
"Memory Used": N, +
"Memory Allocated": N +
}, +
@@ -680,7 +700,7 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
(3 rows)
-- Test SERIALIZE option
-select explain_filter('explain (analyze,serialize) select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
explain_filter
-----------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N loops=N)
@@ -708,7 +728,7 @@ select explain_filter('explain (analyze,serialize binary,buffers,timing) select
(4 rows)
-- this tests an edge case where we have no data to return
-select explain_filter('explain (analyze,serialize) create temp table explain_temp as select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,serialize) create temp table explain_temp as select * from int8_tbl i8');
explain_filter
-----------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N loops=N)
@@ -718,7 +738,7 @@ select explain_filter('explain (analyze,serialize) create temp table explain_tem
(4 rows)
-- Test tuplestore storage usage in Window aggregate (memory case)
-select explain_filter('explain (analyze,costs off) select sum(n) over() from generate_series(1,10) a(n)');
+select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
explain_filter
--------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N loops=N)
@@ -730,7 +750,7 @@ select explain_filter('explain (analyze,costs off) select sum(n) over() from gen
-- Test tuplestore storage usage in Window aggregate (disk case)
set work_mem to 64;
-select explain_filter('explain (analyze,costs off) select sum(n) over() from generate_series(1,2000) a(n)');
+select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,2000) a(n)');
explain_filter
--------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N loops=N)
@@ -741,7 +761,7 @@ select explain_filter('explain (analyze,costs off) select sum(n) over() from gen
(5 rows)
-- Test tuplestore storage usage in Window aggregate (memory and disk case, final result is disk)
-select explain_filter('explain (analyze,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2000) a(n))');
+select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2000) a(n))');
explain_filter
--------------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N loops=N)
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 2df7a5db12..d597575840 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -39,7 +39,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -55,7 +55,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/expected/memoize.out b/src/test/regress/expected/memoize.out
index f6b8329cd6..5ecf971dad 100644
--- a/src/test/regress/expected/memoize.out
+++ b/src/test/regress/expected/memoize.out
@@ -10,7 +10,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/expected/merge.out b/src/test/regress/expected/merge.out
index 521d70a891..28d8551063 100644
--- a/src/test/regress/expected/merge.out
+++ b/src/test/regress/expected/merge.out
@@ -1621,7 +1621,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 7a03b4e360..c52bc40e81 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
@@ -2127,7 +2127,7 @@ create table ab_a3_b3 partition of ab_a3 for values in (3);
set enable_indexonlyscan = off;
prepare ab_q1 (int, int, int) as
select * from ab where a between $1 and $2 and b <= $3;
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2, 3);
QUERY PLAN
---------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2140,7 +2140,7 @@ explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 2, 3);
Filter: ((a >= $1) AND (a <= $2) AND (b <= $3))
(8 rows)
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (1, 2, 3);
QUERY PLAN
---------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2163,7 +2163,7 @@ deallocate ab_q1;
-- Runtime pruning after optimizer pruning
prepare ab_q1 (int, int) as
select a from ab where a between $1 and $2 and b < 3;
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2);
QUERY PLAN
---------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2174,7 +2174,7 @@ explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 2);
Filter: ((a >= $1) AND (a <= $2) AND (b < 3))
(6 rows)
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 4);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 4);
QUERY PLAN
---------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2193,7 +2193,7 @@ explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 4);
-- different levels of partitioning.
prepare ab_q2 (int, int) as
select a from ab where a between $1 and $2 and b < (select 3);
-explain (analyze, costs off, summary off, timing off) execute ab_q2 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q2 (2, 2);
QUERY PLAN
-----------------------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2211,7 +2211,7 @@ explain (analyze, costs off, summary off, timing off) execute ab_q2 (2, 2);
-- As above, but swap the PARAM_EXEC Param to the first partition level
prepare ab_q3 (int, int) as
select a from ab where b between $1 and $2 and a < (select 3);
-explain (analyze, costs off, summary off, timing off) execute ab_q3 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q3 (2, 2);
QUERY PLAN
-----------------------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2273,7 +2273,7 @@ begin;
-- Test run-time pruning using stable functions
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
QUERY PLAN
------------------------------------------------------------------
Append (actual rows=1 loops=1)
@@ -2283,7 +2283,7 @@ explain (analyze, costs off, summary off, timing off) select * from list_part wh
(4 rows)
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
QUERY PLAN
------------------------------------------------------------------
Append (actual rows=4 loops=1)
@@ -2298,7 +2298,7 @@ explain (analyze, costs off, summary off, timing off) select * from list_part wh
(9 rows)
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
QUERY PLAN
------------------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2334,7 +2334,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -2641,7 +2641,7 @@ reset parallel_tuple_cost;
reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
QUERY PLAN
-------------------------------------------------------------------------
@@ -2700,7 +2700,7 @@ select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1
(52 rows)
-- Test run-time partition pruning with UNION ALL parents
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from (select * from ab where a = 1 union all select * from ab) ab where b = (select 1);
QUERY PLAN
-------------------------------------------------------------------------------
@@ -2744,7 +2744,7 @@ select * from (select * from ab where a = 1 union all select * from ab) ab where
(37 rows)
-- A case containing a UNION ALL with a non-partitioned child.
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from (select * from ab where a = 1 union all (values(10,5)) union all select * from ab) ab where b = (select 1);
QUERY PLAN
-------------------------------------------------------------------------------
@@ -2803,7 +2803,7 @@ union all
select tableoid::regclass,a,b from ab
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
QUERY PLAN
--------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -2952,7 +2952,7 @@ create index tprt6_idx on tprt_6 (col1);
insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
--------------------------------------------------------------------------
@@ -2973,7 +2973,7 @@ select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
Index Cond: (col1 < tbl1.col1)
(15 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
--------------------------------------------------------------------------
@@ -3018,7 +3018,7 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
--------------------------------------------------------------------------
@@ -3039,7 +3039,7 @@ select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
Index Cond: (col1 < tbl1.col1)
(15 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
--------------------------------------------------------------------------
@@ -3103,7 +3103,7 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
QUERY PLAN
--------------------------------------------------------------------------
@@ -3135,7 +3135,7 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-------------------------------------------------------------------
@@ -3175,7 +3175,7 @@ alter table part_cab attach partition part_abc_p1 for values in(3);
prepare part_abc_q1 (int, int, int) as
select * from part_abc where a = $1 and b = $2 and c = $3;
-- Single partition should be scanned.
-explain (analyze, costs off, summary off, timing off) execute part_abc_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off) execute part_abc_q1 (1, 2, 3);
QUERY PLAN
----------------------------------------------------------
Seq Scan on part_abc_p1 part_abc (actual rows=0 loops=1)
@@ -3200,7 +3200,7 @@ select * from listp where b = 1;
-- partitions before finally detecting the correct set of 2nd level partitions
-- which match the given parameter.
prepare q1 (int,int) as select * from listp where b in ($1,$2);
-explain (analyze, costs off, summary off, timing off) execute q1 (1,1);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,1);
QUERY PLAN
-------------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -3209,7 +3209,7 @@ explain (analyze, costs off, summary off, timing off) execute q1 (1,1);
Filter: (b = ANY (ARRAY[$1, $2]))
(4 rows)
-explain (analyze, costs off, summary off, timing off) execute q1 (2,2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (2,2);
QUERY PLAN
-------------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -3219,7 +3219,7 @@ explain (analyze, costs off, summary off, timing off) execute q1 (2,2);
(4 rows)
-- Try with no matching partitions.
-explain (analyze, costs off, summary off, timing off) execute q1 (0,0);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (0,0);
QUERY PLAN
--------------------------------
Append (actual rows=0 loops=1)
@@ -3230,7 +3230,7 @@ deallocate q1;
-- Test more complex cases where a not-equal condition further eliminates partitions.
prepare q1 (int,int,int,int) as select * from listp where b in($1,$2) and $3 <> b and $4 <> b;
-- Both partitions allowed by IN clause, but one disallowed by <> clause
-explain (analyze, costs off, summary off, timing off) execute q1 (1,2,2,0);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,0);
QUERY PLAN
-------------------------------------------------------------------------
Append (actual rows=0 loops=1)
@@ -3240,7 +3240,7 @@ explain (analyze, costs off, summary off, timing off) execute q1 (1,2,2,0);
(4 rows)
-- Both partitions allowed by IN clause, then both excluded again by <> clauses.
-explain (analyze, costs off, summary off, timing off) execute q1 (1,2,2,1);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,1);
QUERY PLAN
--------------------------------
Append (actual rows=0 loops=1)
@@ -3248,7 +3248,7 @@ explain (analyze, costs off, summary off, timing off) execute q1 (1,2,2,1);
(2 rows)
-- Ensure Params that evaluate to NULL properly prune away all partitions
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from listp where a = (select null::int);
QUERY PLAN
------------------------------------------------------
@@ -3273,7 +3273,7 @@ create table stable_qual_pruning2 partition of stable_qual_pruning
create table stable_qual_pruning3 partition of stable_qual_pruning
for values from ('3000-02-01') to ('3000-03-01');
-- comparison against a stable value requires run-time pruning
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning where a < localtimestamp;
QUERY PLAN
--------------------------------------------------------------------------------------
@@ -3286,7 +3286,7 @@ select * from stable_qual_pruning where a < localtimestamp;
(6 rows)
-- timestamp < timestamptz comparison is only stable, not immutable
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning where a < '2000-02-01'::timestamptz;
QUERY PLAN
--------------------------------------------------------------------------------------
@@ -3297,7 +3297,7 @@ select * from stable_qual_pruning where a < '2000-02-01'::timestamptz;
(4 rows)
-- check ScalarArrayOp cases
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamp[]);
QUERY PLAN
@@ -3306,7 +3306,7 @@ select * from stable_qual_pruning
One-Time Filter: false
(2 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamp[]);
QUERY PLAN
@@ -3315,7 +3315,7 @@ select * from stable_qual_pruning
Filter: (a = ANY ('{"Tue Feb 01 00:00:00 2000","Fri Jan 01 00:00:00 2010"}'::timestamp without time zone[]))
(2 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', localtimestamp]::timestamp[]);
QUERY PLAN
@@ -3326,7 +3326,7 @@ select * from stable_qual_pruning
Filter: (a = ANY (ARRAY['Tue Feb 01 00:00:00 2000'::timestamp without time zone, LOCALTIMESTAMP]))
(4 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamptz[]);
QUERY PLAN
@@ -3335,7 +3335,7 @@ select * from stable_qual_pruning
Subplans Removed: 3
(2 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamptz[]);
QUERY PLAN
@@ -3346,7 +3346,7 @@ select * from stable_qual_pruning
Filter: (a = ANY ('{"Tue Feb 01 00:00:00 2000 PST","Fri Jan 01 00:00:00 2010 PST"}'::timestamp with time zone[]))
(4 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(null::timestamptz[]);
QUERY PLAN
@@ -3374,7 +3374,7 @@ create table mc3p1 partition of mc3p
create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from mc3p where a < 3 and abs(b) = 1;
QUERY PLAN
--------------------------------------------------------
@@ -3394,7 +3394,7 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
execute ps1(1);
QUERY PLAN
-------------------------------------------------------------
@@ -3409,7 +3409,7 @@ execute ps1(1);
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
execute ps2(1);
QUERY PLAN
--------------------------------------------------------------
@@ -3431,7 +3431,7 @@ insert into boolvalues values('t'),('f');
create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from boolp where a = (select value from boolvalues where value);
QUERY PLAN
-----------------------------------------------------------
@@ -3446,7 +3446,7 @@ select * from boolp where a = (select value from boolvalues where value);
Filter: (a = (InitPlan 1).col1)
(9 rows)
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from boolp where a = (select value from boolvalues where not value);
QUERY PLAN
-----------------------------------------------------------
@@ -3475,7 +3475,7 @@ insert into ma_test select x,x from generate_series(0,29) t(x);
create index on ma_test (b);
analyze ma_test;
prepare mt_q1 (int) as select a from ma_test where a >= $1 and a % 10 = 5 order by b;
-explain (analyze, costs off, summary off, timing off) execute mt_q1(15);
+explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(15);
QUERY PLAN
-----------------------------------------------------------------------------------------
Merge Append (actual rows=2 loops=1)
@@ -3496,7 +3496,7 @@ execute mt_q1(15);
25
(2 rows)
-explain (analyze, costs off, summary off, timing off) execute mt_q1(25);
+explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(25);
QUERY PLAN
-----------------------------------------------------------------------------------------
Merge Append (actual rows=1 loops=1)
@@ -3514,7 +3514,7 @@ execute mt_q1(25);
(1 row)
-- Ensure MergeAppend behaves correctly when no subplans match
-explain (analyze, costs off, summary off, timing off) execute mt_q1(35);
+explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(35);
QUERY PLAN
--------------------------------------
Merge Append (actual rows=0 loops=1)
@@ -3530,7 +3530,7 @@ execute mt_q1(35);
deallocate mt_q1;
prepare mt_q2 (int) as select * from ma_test where a >= $1 order by b limit 1;
-- Ensure output list looks sane when the MergeAppend has no subplans.
-explain (analyze, verbose, costs off, summary off, timing off) execute mt_q2 (35);
+explain (analyze, verbose, costs off, summary off, timing off, buffers off) execute mt_q2 (35);
QUERY PLAN
--------------------------------------------
Limit (actual rows=0 loops=1)
@@ -3542,7 +3542,7 @@ explain (analyze, verbose, costs off, summary off, timing off) execute mt_q2 (35
deallocate mt_q2;
-- ensure initplan params properly prune partitions
-explain (analyze, costs off, summary off, timing off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
+explain (analyze, costs off, summary off, timing off, buffers off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
QUERY PLAN
-----------------------------------------------------------------------------------------------
Merge Append (actual rows=20 loops=1)
@@ -3992,7 +3992,7 @@ create table listp (a int, b int) partition by list (a);
create table listp1 partition of listp for values in(1);
create table listp2 partition of listp for values in(2) partition by list(b);
create table listp2_10 partition of listp2 for values in (10);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from listp where a = (select 2) and b <> 10;
QUERY PLAN
---------------------------------------------------
@@ -4117,7 +4117,7 @@ create table rangep_0_to_100_3 partition of rangep_0_to_100 for values in(3);
create table rangep_100_to_200 partition of rangep for values from (100) to (200);
create index on rangep (a);
-- Ensure run-time pruning works on the nested Merge Append
-explain (analyze on, costs off, timing off, summary off)
+explain (analyze on, costs off, timing off, summary off, buffers off)
select * from rangep where b IN((select 1),(select 2)) order by a;
QUERY PLAN
------------------------------------------------------------------------------------------------------------
diff --git a/src/test/regress/expected/select.out b/src/test/regress/expected/select.out
index 33a6dceb0e..88911ca2b9 100644
--- a/src/test/regress/expected/select.out
+++ b/src/test/regress/expected/select.out
@@ -757,7 +757,7 @@ select * from onek2 where unique2 = 11 and stringu1 = 'ATAAAA';
(1 row)
-- actually run the query with an analyze to use the partial index
-explain (costs off, analyze on, timing off, summary off)
+explain (costs off, analyze on, timing off, summary off, buffers off)
select * from onek2 where unique2 = 11 and stringu1 = 'ATAAAA';
QUERY PLAN
-----------------------------------------------------------------
diff --git a/src/test/regress/expected/select_into.out b/src/test/regress/expected/select_into.out
index b79fe9a1c0..d6578d992c 100644
--- a/src/test/regress/expected/select_into.out
+++ b/src/test/regress/expected/select_into.out
@@ -25,7 +25,7 @@ CREATE TABLE selinto_schema.tbl_withdata1 (a)
AS SELECT generate_series(1,3) WITH DATA;
INSERT INTO selinto_schema.tbl_withdata1 VALUES (4);
ERROR: permission denied for table tbl_withdata1
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
CREATE TABLE selinto_schema.tbl_withdata2 (a) AS
SELECT generate_series(1,3) WITH DATA;
QUERY PLAN
@@ -62,7 +62,7 @@ EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
-- EXECUTE and WITH NO DATA, passes.
CREATE TABLE selinto_schema.tbl_nodata3 (a) AS
EXECUTE data_sel WITH NO DATA;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
CREATE TABLE selinto_schema.tbl_nodata4 (a) AS
EXECUTE data_sel WITH NO DATA;
QUERY PLAN
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 8c31f6460d..a809036453 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -580,7 +580,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off)
+explain (analyze, timing off, summary off, costs off, buffers off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
QUERY PLAN
@@ -606,7 +606,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off)
+ explain (analyze, timing off, summary off, costs off, buffers off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -1169,7 +1169,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
QUERY PLAN
-------------------------------------------------------------
Gather (actual rows=10000 loops=1)
diff --git a/src/test/regress/expected/subselect.out b/src/test/regress/expected/subselect.out
index 5c5a769b55..b997eb7690 100644
--- a/src/test/regress/expected/subselect.out
+++ b/src/test/regress/expected/subselect.out
@@ -1675,7 +1675,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off)
+ explain (analyze, summary off, timing off, costs off, buffers off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
diff --git a/src/test/regress/expected/tidscan.out b/src/test/regress/expected/tidscan.out
index f133b5a4ac..f6ebdf0601 100644
--- a/src/test/regress/expected/tidscan.out
+++ b/src/test/regress/expected/tidscan.out
@@ -189,7 +189,7 @@ FETCH NEXT FROM c;
(1 row)
-- perform update
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
UPDATE tidscan SET id = -id WHERE CURRENT OF c RETURNING *;
QUERY PLAN
---------------------------------------------------
@@ -205,7 +205,7 @@ FETCH NEXT FROM c;
(1 row)
-- perform update
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
UPDATE tidscan SET id = -id WHERE CURRENT OF c RETURNING *;
QUERY PLAN
---------------------------------------------------
@@ -229,7 +229,7 @@ FETCH NEXT FROM c;
(0 rows)
-- should error out
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
UPDATE tidscan SET id = -id WHERE CURRENT OF c RETURNING *;
ERROR: cursor "c" is not positioned on a row
ROLLBACK;
diff --git a/src/test/regress/sql/brin_multi.sql b/src/test/regress/sql/brin_multi.sql
index 55349b4e1f..dda9fb73e0 100644
--- a/src/test/regress/sql/brin_multi.sql
+++ b/src/test/regress/sql/brin_multi.sql
@@ -619,7 +619,7 @@ CREATE INDEX ON brin_date_test USING brin (a date_minmax_multi_ops) WITH (pages_
SET enable_seqscan = off;
-- make sure the ranges were built correctly and 2023-01-01 eliminates all
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_date_test WHERE a = '2023-01-01'::date;
DROP TABLE brin_date_test;
@@ -636,10 +636,10 @@ CREATE INDEX ON brin_timestamp_test USING brin (a timestamp_minmax_multi_ops) WI
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_timestamp_test WHERE a = '2023-01-01'::timestamp;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_timestamp_test WHERE a = '1900-01-01'::timestamp;
DROP TABLE brin_timestamp_test;
@@ -655,10 +655,10 @@ CREATE INDEX ON brin_date_test USING brin (a date_minmax_multi_ops) WITH (pages_
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_date_test WHERE a = '2023-01-01'::date;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_date_test WHERE a = '1900-01-01'::date;
DROP TABLE brin_date_test;
@@ -676,10 +676,10 @@ CREATE INDEX ON brin_interval_test USING brin (a interval_minmax_multi_ops) WITH
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '-30 years'::interval;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '30 years'::interval;
DROP TABLE brin_interval_test;
@@ -695,10 +695,10 @@ CREATE INDEX ON brin_interval_test USING brin (a interval_minmax_multi_ops) WITH
SET enable_seqscan = off;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '-30 years'::interval;
-EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF)
+EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF, SUMMARY OFF, BUFFERS OFF)
SELECT * FROM brin_interval_test WHERE a = '30 years'::interval;
DROP TABLE brin_interval_test;
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index 3ca285a1d7..0bafa87049 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -62,8 +62,8 @@ set track_io_timing = off;
-- Simple cases
select explain_filter('explain select * from int8_tbl i8');
-select explain_filter('explain (analyze) select * from int8_tbl i8');
-select explain_filter('explain (analyze, verbose) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
@@ -96,7 +96,7 @@ select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1
-- MEMORY option
select explain_filter('explain (memory) select * from int8_tbl i8');
-select explain_filter('explain (memory, analyze) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, buffers off) select * from int8_tbl i8');
select explain_filter('explain (memory, summary, format yaml) select * from int8_tbl i8');
select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
prepare int8_query as select * from int8_tbl i8;
@@ -168,17 +168,17 @@ select explain_filter('explain (verbose) declare test_cur cursor for select * fr
select explain_filter('explain (verbose) create table test_ctas as select 1');
-- Test SERIALIZE option
-select explain_filter('explain (analyze,serialize) select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
-- this tests an edge case where we have no data to return
-select explain_filter('explain (analyze,serialize) create temp table explain_temp as select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,serialize) create temp table explain_temp as select * from int8_tbl i8');
-- Test tuplestore storage usage in Window aggregate (memory case)
-select explain_filter('explain (analyze,costs off) select sum(n) over() from generate_series(1,10) a(n)');
+select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
-- Test tuplestore storage usage in Window aggregate (disk case)
set work_mem to 64;
-select explain_filter('explain (analyze,costs off) select sum(n) over() from generate_series(1,2000) a(n)');
+select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,2000) a(n)');
-- Test tuplestore storage usage in Window aggregate (memory and disk case, final result is disk)
-select explain_filter('explain (analyze,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2000) a(n))');
+select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2000) a(n))');
reset work_mem;
diff --git a/src/test/regress/sql/incremental_sort.sql b/src/test/regress/sql/incremental_sort.sql
index 98b20e17e1..f1f8fae565 100644
--- a/src/test/regress/sql/incremental_sort.sql
+++ b/src/test/regress/sql/incremental_sort.sql
@@ -21,7 +21,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -38,7 +38,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/sql/memoize.sql b/src/test/regress/sql/memoize.sql
index 2eaeb1477a..d5aab4e566 100644
--- a/src/test/regress/sql/memoize.sql
+++ b/src/test/regress/sql/memoize.sql
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/sql/merge.sql b/src/test/regress/sql/merge.sql
index 5ddcca84f8..54929a92fa 100644
--- a/src/test/regress/sql/merge.sql
+++ b/src/test/regress/sql/merge.sql
@@ -1072,7 +1072,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index 442428d937..d67598d5c7 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -12,7 +12,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
@@ -465,8 +465,8 @@ set enable_indexonlyscan = off;
prepare ab_q1 (int, int, int) as
select * from ab where a between $1 and $2 and b <= $3;
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 2, 3);
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (1, 2, 3);
deallocate ab_q1;
@@ -474,21 +474,21 @@ deallocate ab_q1;
prepare ab_q1 (int, int) as
select a from ab where a between $1 and $2 and b < 3;
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 2);
-explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 4);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 4);
-- Ensure a mix of PARAM_EXTERN and PARAM_EXEC Params work together at
-- different levels of partitioning.
prepare ab_q2 (int, int) as
select a from ab where a between $1 and $2 and b < (select 3);
-explain (analyze, costs off, summary off, timing off) execute ab_q2 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q2 (2, 2);
-- As above, but swap the PARAM_EXEC Param to the first partition level
prepare ab_q3 (int, int) as
select a from ab where b between $1 and $2 and a < (select 3);
-explain (analyze, costs off, summary off, timing off) execute ab_q3 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q3 (2, 2);
--
-- Test runtime pruning with hash partitioned tables
@@ -538,13 +538,13 @@ begin;
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
rollback;
@@ -567,7 +567,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -650,15 +650,15 @@ reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
-- Test run-time partition pruning with UNION ALL parents
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from (select * from ab where a = 1 union all select * from ab) ab where b = (select 1);
-- A case containing a UNION ALL with a non-partitioned child.
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from (select * from ab where a = 1 union all (values(10,5)) union all select * from ab) ab where b = (select 1);
-- Another UNION ALL test, but containing a mix of exec init and exec run-time pruning.
@@ -678,7 +678,7 @@ union all
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
-- Ensure we see just the xy_1 row.
execute ab_q6(100);
@@ -733,10 +733,10 @@ insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -749,10 +749,10 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -766,7 +766,7 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -776,7 +776,7 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -799,7 +799,7 @@ prepare part_abc_q1 (int, int, int) as
select * from part_abc where a = $1 and b = $2 and c = $3;
-- Single partition should be scanned.
-explain (analyze, costs off, summary off, timing off) execute part_abc_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off) execute part_abc_q1 (1, 2, 3);
deallocate part_abc_q1;
@@ -819,12 +819,12 @@ select * from listp where b = 1;
-- which match the given parameter.
prepare q1 (int,int) as select * from listp where b in ($1,$2);
-explain (analyze, costs off, summary off, timing off) execute q1 (1,1);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,1);
-explain (analyze, costs off, summary off, timing off) execute q1 (2,2);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (2,2);
-- Try with no matching partitions.
-explain (analyze, costs off, summary off, timing off) execute q1 (0,0);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (0,0);
deallocate q1;
@@ -832,13 +832,13 @@ deallocate q1;
prepare q1 (int,int,int,int) as select * from listp where b in($1,$2) and $3 <> b and $4 <> b;
-- Both partitions allowed by IN clause, but one disallowed by <> clause
-explain (analyze, costs off, summary off, timing off) execute q1 (1,2,2,0);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,0);
-- Both partitions allowed by IN clause, then both excluded again by <> clauses.
-explain (analyze, costs off, summary off, timing off) execute q1 (1,2,2,1);
+explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,1);
-- Ensure Params that evaluate to NULL properly prune away all partitions
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from listp where a = (select null::int);
drop table listp;
@@ -855,30 +855,30 @@ create table stable_qual_pruning3 partition of stable_qual_pruning
for values from ('3000-02-01') to ('3000-03-01');
-- comparison against a stable value requires run-time pruning
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning where a < localtimestamp;
-- timestamp < timestamptz comparison is only stable, not immutable
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning where a < '2000-02-01'::timestamptz;
-- check ScalarArrayOp cases
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamp[]);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamp[]);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', localtimestamp]::timestamp[]);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamptz[]);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamptz[]);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from stable_qual_pruning
where a = any(null::timestamptz[]);
@@ -898,7 +898,7 @@ create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from mc3p where a < 3 and abs(b) = 1;
--
@@ -908,12 +908,12 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
execute ps1(1);
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
execute ps2(1);
deallocate ps2;
@@ -927,10 +927,10 @@ create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from boolp where a = (select value from boolvalues where value);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from boolp where a = (select value from boolvalues where not value);
drop table boolp;
@@ -950,12 +950,12 @@ create index on ma_test (b);
analyze ma_test;
prepare mt_q1 (int) as select a from ma_test where a >= $1 and a % 10 = 5 order by b;
-explain (analyze, costs off, summary off, timing off) execute mt_q1(15);
+explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(15);
execute mt_q1(15);
-explain (analyze, costs off, summary off, timing off) execute mt_q1(25);
+explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(25);
execute mt_q1(25);
-- Ensure MergeAppend behaves correctly when no subplans match
-explain (analyze, costs off, summary off, timing off) execute mt_q1(35);
+explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(35);
execute mt_q1(35);
deallocate mt_q1;
@@ -963,12 +963,12 @@ deallocate mt_q1;
prepare mt_q2 (int) as select * from ma_test where a >= $1 order by b limit 1;
-- Ensure output list looks sane when the MergeAppend has no subplans.
-explain (analyze, verbose, costs off, summary off, timing off) execute mt_q2 (35);
+explain (analyze, verbose, costs off, summary off, timing off, buffers off) execute mt_q2 (35);
deallocate mt_q2;
-- ensure initplan params properly prune partitions
-explain (analyze, costs off, summary off, timing off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
+explain (analyze, costs off, summary off, timing off, buffers off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
reset enable_seqscan;
reset enable_sort;
@@ -1148,7 +1148,7 @@ create table listp1 partition of listp for values in(1);
create table listp2 partition of listp for values in(2) partition by list(b);
create table listp2_10 partition of listp2 for values in (10);
-explain (analyze, costs off, summary off, timing off)
+explain (analyze, costs off, summary off, timing off, buffers off)
select * from listp where a = (select 2) and b <> 10;
--
@@ -1216,7 +1216,7 @@ create table rangep_100_to_200 partition of rangep for values from (100) to (200
create index on rangep (a);
-- Ensure run-time pruning works on the nested Merge Append
-explain (analyze on, costs off, timing off, summary off)
+explain (analyze on, costs off, timing off, summary off, buffers off)
select * from rangep where b IN((select 1),(select 2)) order by a;
reset enable_sort;
drop table rangep;
diff --git a/src/test/regress/sql/select.sql b/src/test/regress/sql/select.sql
index 019f1e7673..1d1bf2b931 100644
--- a/src/test/regress/sql/select.sql
+++ b/src/test/regress/sql/select.sql
@@ -196,7 +196,7 @@ explain (costs off)
select * from onek2 where unique2 = 11 and stringu1 = 'ATAAAA';
select * from onek2 where unique2 = 11 and stringu1 = 'ATAAAA';
-- actually run the query with an analyze to use the partial index
-explain (costs off, analyze on, timing off, summary off)
+explain (costs off, analyze on, timing off, summary off, buffers off)
select * from onek2 where unique2 = 11 and stringu1 = 'ATAAAA';
explain (costs off)
select unique2 from onek2 where unique2 = 11 and stringu1 = 'ATAAAA';
diff --git a/src/test/regress/sql/select_into.sql b/src/test/regress/sql/select_into.sql
index 689c448cc2..106cdde187 100644
--- a/src/test/regress/sql/select_into.sql
+++ b/src/test/regress/sql/select_into.sql
@@ -30,7 +30,7 @@ SET SESSION AUTHORIZATION regress_selinto_user;
CREATE TABLE selinto_schema.tbl_withdata1 (a)
AS SELECT generate_series(1,3) WITH DATA;
INSERT INTO selinto_schema.tbl_withdata1 VALUES (4);
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
CREATE TABLE selinto_schema.tbl_withdata2 (a) AS
SELECT generate_series(1,3) WITH DATA;
-- WITH NO DATA, passes.
@@ -49,7 +49,7 @@ EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
-- EXECUTE and WITH NO DATA, passes.
CREATE TABLE selinto_schema.tbl_nodata3 (a) AS
EXECUTE data_sel WITH NO DATA;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
CREATE TABLE selinto_schema.tbl_nodata4 (a) AS
EXECUTE data_sel WITH NO DATA;
RESET SESSION AUTHORIZATION;
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 5b4a6e1088..71a75bc86e 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -230,7 +230,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off)
+explain (analyze, timing off, summary off, costs off, buffers off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
alter table tenk2 reset (parallel_workers);
@@ -242,7 +242,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off)
+ explain (analyze, timing off, summary off, costs off, buffers off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -450,7 +450,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
ROLLBACK TO SAVEPOINT settings;
-- provoke error in worker
diff --git a/src/test/regress/sql/subselect.sql b/src/test/regress/sql/subselect.sql
index db1969256f..af5a1d8b55 100644
--- a/src/test/regress/sql/subselect.sql
+++ b/src/test/regress/sql/subselect.sql
@@ -857,7 +857,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off)
+ explain (analyze, summary off, timing off, costs off, buffers off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
diff --git a/src/test/regress/sql/tidscan.sql b/src/test/regress/sql/tidscan.sql
index 313e0fb9b6..1b82d5f1a5 100644
--- a/src/test/regress/sql/tidscan.sql
+++ b/src/test/regress/sql/tidscan.sql
@@ -68,17 +68,17 @@ DECLARE c CURSOR FOR SELECT ctid, * FROM tidscan;
FETCH NEXT FROM c; -- skip one row
FETCH NEXT FROM c;
-- perform update
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
UPDATE tidscan SET id = -id WHERE CURRENT OF c RETURNING *;
FETCH NEXT FROM c;
-- perform update
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
UPDATE tidscan SET id = -id WHERE CURRENT OF c RETURNING *;
SELECT * FROM tidscan;
-- position cursor past any rows
FETCH NEXT FROM c;
-- should error out
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
UPDATE tidscan SET id = -id WHERE CURRENT OF c RETURNING *;
ROLLBACK;
--
2.34.1
[application/octet-stream] v5-0002-Enable-auto_explain.log_buffers-by-default.patch (2.1K, ../../CAApHDvpj0Cj6Z8VnDN5htzF24b4_hKL+1_n=HOehLHkBK_E_TQ@mail.gmail.com/3-v5-0002-Enable-auto_explain.log_buffers-by-default.patch)
download | inline diff:
From 07366a0b8cb4ef30a811e4ee159f4161b8d7ade1 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 10 Dec 2024 14:44:58 +1300
Subject: [PATCH v5 2/2] Enable auto_explain.log_buffers by default
Now that EXPLAIN ANALAYZE causes BUFFERS to be enabled by default,
adjust auto_explain.log_buffers to default to on.
Author: Guillaume Lelarge
Reviewed-by: Robert Haas
Discussion: https://postgr.es/m/CA+TgmoZO2HtDGHf7K4rmNdfUGuS=ngsPgwrH=JdyF6Okcnzfkw@mail.gmail.com
---
contrib/auto_explain/auto_explain.c | 4 ++--
doc/src/sgml/auto-explain.sgml | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index f2eaa8e494..17abd24ea1 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -27,7 +27,7 @@ static int auto_explain_log_min_duration = -1; /* msec or -1 */
static int auto_explain_log_parameter_max_length = -1; /* bytes or -1 */
static bool auto_explain_log_analyze = false;
static bool auto_explain_log_verbose = false;
-static bool auto_explain_log_buffers = false;
+static bool auto_explain_log_buffers = true;
static bool auto_explain_log_wal = false;
static bool auto_explain_log_triggers = false;
static bool auto_explain_log_timing = true;
@@ -152,7 +152,7 @@ _PG_init(void)
"Log buffers usage.",
NULL,
&auto_explain_log_buffers,
- false,
+ true,
PGC_SUSET,
0,
NULL,
diff --git a/doc/src/sgml/auto-explain.sgml b/doc/src/sgml/auto-explain.sgml
index 0c4656ee30..6623d36c99 100644
--- a/doc/src/sgml/auto-explain.sgml
+++ b/doc/src/sgml/auto-explain.sgml
@@ -122,7 +122,7 @@ LOAD 'auto_explain';
equivalent to the <literal>BUFFERS</literal> option of <command>EXPLAIN</command>.
This parameter has no effect
unless <varname>auto_explain.log_analyze</varname> is enabled.
- This parameter is off by default.
+ This parameter is on by default.
Only superusers can change this setting.
</para>
</listitem>
--
2.34.1
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-12-10 12:22 Guillaume Lelarge <[email protected]>
parent: David Rowley <[email protected]>
0 siblings, 1 reply; 32+ messages in thread
From: Guillaume Lelarge @ 2024-12-10 12:22 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
Hello,
Le mar. 10 déc. 2024 à 03:57, David Rowley <[email protected]> a écrit :
> On Thu, 21 Nov 2024 at 08:30, Guillaume Lelarge <[email protected]>
> wrote:
> > OK, I'm fine with this. v4 patch attached with one plan showing read,
> written, and dirtied buffers.
>
> Today I spent more time polishing this patch. There were a few cases
> in the docs that displayed EXPLAIN ANALYZE output that you'd not
> adjusted to include the buffers output or adjusted to do BUFFERS OFF.
> I think I've got all these now. Tom went to some effort to fix some
> outdated EXPLAIN outputs for v17 in 5caa05749, so I think we owe it to
> him not to let these go out of date so soon after that change.
>
>
You're right and I completely forgot to check the whole documentation. I
just looked at perform.sgml which was the obvious file for explain plans.
Anyway, sorry about this, and thanks a lot for your work on this patch.
I also was thinking again about what Robert mentioned about
> auto_explain.log_buffers should now also be on by default. I'm less
> certain than him about this change. It seems like a separate
> consideration that we could apply many of the same arguments for the
> main change to. In any case, I extracted that change from the 0001
> patch and put it in a 0002 patch as it doesn't seem like something
> that should be a sidenote in the commit message. I felt doing that
> increases the chances that it would be overlooked in the release
> notes.
>
> I was very close to pushing 0001 today, but will hold off until
> tomorrow to see if anyone has final comments.
>
>
No more comments. I'm fine with both patches.
> For 0002, I'd really like to see a bit more justification for it. For
> the record, I'm not against 0002, it's just that my personal arguments
> for wanting 0001 don't apply to 0002.
>
>
I guess consistency is the key word here. But I agree that 0001 is the one
that's really important to me.
Thanks again for your work on this.
--
Guillaume.
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-12-11 09:38 David Rowley <[email protected]>
parent: Guillaume Lelarge <[email protected]>
0 siblings, 2 replies; 32+ messages in thread
From: David Rowley @ 2024-12-11 09:38 UTC (permalink / raw)
To: Guillaume Lelarge <[email protected]>; +Cc: Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Wed, 11 Dec 2024 at 01:22, Guillaume Lelarge <[email protected]> wrote:
> Le mar. 10 déc. 2024 à 03:57, David Rowley <[email protected]> a écrit :
>> I was very close to pushing 0001 today, but will hold off until
>> tomorrow to see if anyone has final comments.
>>
> No more comments. I'm fine with both patches.
Thanks for looking.
I've pushed the main patch. I'm not planning on pushing the
auto_explain.log_buffers default change unless there's a bit more
discussion about it.
David
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-12-11 11:16 David Rowley <[email protected]>
parent: David Rowley <[email protected]>
1 sibling, 3 replies; 32+ messages in thread
From: David Rowley @ 2024-12-11 11:16 UTC (permalink / raw)
To: Guillaume Lelarge <[email protected]>; +Cc: Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Wed, 11 Dec 2024 at 22:38, David Rowley <[email protected]> wrote:
> I've pushed the main patch. I'm not planning on pushing the
> auto_explain.log_buffers default change unless there's a bit more
> discussion about it.
There were a few places that were missing a BUFFERS OFF as indicated
by the buildfarm. I've fixed a few of these, but there's at least one
more in pg_stat_statements's level_tracking.sql. I'm currently running
a -DRELCACHE_FORCE_RELEASE -DCATCACHE_FORCE_RELEASE enabled make
check-world. It's going to take a while, so I'll hold off doing yet
another commit to fix these until I see if the level_tracking.sql one
is the last issue to fix.
David
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-12-11 12:41 Guillaume Lelarge <[email protected]>
parent: David Rowley <[email protected]>
2 siblings, 0 replies; 32+ messages in thread
From: Guillaume Lelarge @ 2024-12-11 12:41 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
Le mer. 11 déc. 2024 à 12:16, David Rowley <[email protected]> a écrit :
> On Wed, 11 Dec 2024 at 22:38, David Rowley <[email protected]> wrote:
> > I've pushed the main patch. I'm not planning on pushing the
> > auto_explain.log_buffers default change unless there's a bit more
> > discussion about it.
>
> There were a few places that were missing a BUFFERS OFF as indicated
> by the buildfarm. I've fixed a few of these, but there's at least one
> more in pg_stat_statements's level_tracking.sql. I'm currently running
> a -DRELCACHE_FORCE_RELEASE -DCATCACHE_FORCE_RELEASE enabled make
> check-world. It's going to take a while, so I'll hold off doing yet
> another commit to fix these until I see if the level_tracking.sql one
> is the last issue to fix.
>
>
Sorry for all those late fixes. I'll refrain from saying "Sure looks easy
enough to do" from now on :) Guess this is a great example of what Robert
said in
http://rhaas.blogspot.com/2024/05/hacking-on-postgresql-is-really-hard.html
.
Anyway, thanks again for your work on this.
--
Guillaume.
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-12-11 14:00 jian he <[email protected]>
parent: David Rowley <[email protected]>
2 siblings, 1 reply; 32+ messages in thread
From: jian he @ 2024-12-11 14:00 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Guillaume Lelarge <[email protected]>; Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
do we also need to update doc/src/sgml/rules.sgml?
https://www.postgresql.org/docs/current/rules-materializedviews.html
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-12-11 20:29 Guillaume Lelarge <[email protected]>
parent: jian he <[email protected]>
0 siblings, 0 replies; 32+ messages in thread
From: Guillaume Lelarge @ 2024-12-11 20:29 UTC (permalink / raw)
To: jian he <[email protected]>; +Cc: David Rowley <[email protected]>; Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
Le mer. 11 déc. 2024 à 15:00, jian he <[email protected]> a
écrit :
> do we also need to update doc/src/sgml/rules.sgml?
>
> https://www.postgresql.org/docs/current/rules-materializedviews.html
>
Yes, you're right. Here is a fix. (Not a patch with a commit message as I
don't know if David will want to commit everything (the CI issue, this, and
probably more) in one patch.)
--
Guillaume.
Attachments:
[text/x-patch] rules.patch (2.0K, ../../CAECtzeXsJbONQDGu1dC9f_RO2WLFK2ZwPNrFKTvYMbWE+w-oAQ@mail.gmail.com/3-rules.patch)
download | inline diff:
diff --git a/doc/src/sgml/rules.sgml b/doc/src/sgml/rules.sgml
index 7a928bd7b9..6cebed5c37 100644
--- a/doc/src/sgml/rules.sgml
+++ b/doc/src/sgml/rules.sgml
@@ -1042,12 +1042,16 @@ SELECT count(*) FROM words WHERE word = 'caterpiler';
If the materialized view is used instead, the query is much faster:
<programlisting>
- Aggregate (cost=4.44..4.45 rows=1 width=0) (actual time=0.042..0.042 rows=1 loops=1)
- -> Index Only Scan using wrd_word on wrd (cost=0.42..4.44 rows=1 width=0) (actual time=0.039..0.039 rows=0 loops=1)
+ Aggregate (cost=4.44..4.45 rows=1 width=8) (actual time=0.138..0.139 rows=1 loops=1)
+ Buffers: shared read=3
+ -> Index Only Scan using wrd_word on wrd (cost=0.42..4.44 rows=1 width=0) (actual time=0.132..0.133 rows=0 loops=1)
Index Cond: (word = 'caterpiler'::text)
Heap Fetches: 0
- Planning time: 0.164 ms
- Execution time: 0.117 ms
+ Buffers: shared read=3
+ Planning:
+ Buffers: shared hit=5
+ Planning Time: 0.143 ms
+ Execution Time: 0.200 ms
</programlisting>
Either way, the word is spelled wrong, so let's look for what we might
@@ -1087,11 +1091,15 @@ SELECT word FROM words ORDER BY word <-> 'caterpiler' LIMIT 10;
Using the materialized view:
<programlisting>
- Limit (cost=0.29..1.06 rows=10 width=10) (actual time=187.222..188.257 rows=10 loops=1)
- -> Index Scan using wrd_trgm on wrd (cost=0.29..37020.87 rows=479829 width=10) (actual time=187.219..188.252 rows=10 loops=1)
+ Limit (cost=0.29..1.06 rows=10 width=14) (actual time=135.027..135.475 rows=10 loops=1)
+ Buffers: shared hit=4465
+ -> Index Scan using wrd_trgm on wrd (cost=0.29..37224.81 rows=479826 width=14) (actual time=135.023..135.470 rows=10 loops=1)
Order By: (word <-> 'caterpiler'::text)
- Planning time: 0.196 ms
- Execution time: 198.640 ms
+ Buffers: shared hit=4465
+ Planning:
+ Buffers: shared hit=80
+ Planning Time: 0.767 ms
+ Execution Time: 140.836 ms
</programlisting>
If you can tolerate periodic update of the remote data to the local
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-12-11 20:41 Jelte Fennema-Nio <[email protected]>
parent: David Rowley <[email protected]>
1 sibling, 1 reply; 32+ messages in thread
From: Jelte Fennema-Nio @ 2024-12-11 20:41 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Guillaume Lelarge <[email protected]>; Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Wed, 11 Dec 2024 at 10:38, David Rowley <[email protected]> wrote:
> I've pushed the main patch. I'm not planning on pushing the
> auto_explain.log_buffers default change unless there's a bit more
> discussion about it.
FreeBSD CFBot seems broken on a pg_stat_statements test because of
this change: https://api.cirrus-ci.com/v1/artifact/task/4845908522696704/testrun/build/testrun/pg_stat_statements...
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-12-11 20:46 Guillaume Lelarge <[email protected]>
parent: Jelte Fennema-Nio <[email protected]>
0 siblings, 0 replies; 32+ messages in thread
From: Guillaume Lelarge @ 2024-12-11 20:46 UTC (permalink / raw)
To: Jelte Fennema-Nio <[email protected]>; +Cc: David Rowley <[email protected]>; Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
Le mer. 11 déc. 2024 à 21:41, Jelte Fennema-Nio <[email protected]> a
écrit :
> On Wed, 11 Dec 2024 at 10:38, David Rowley <[email protected]> wrote:
> > I've pushed the main patch. I'm not planning on pushing the
> > auto_explain.log_buffers default change unless there's a bit more
> > discussion about it.
>
> FreeBSD CFBot seems broken on a pg_stat_statements test because of
> this change:
> https://api.cirrus-ci.com/v1/artifact/task/4845908522696704/testrun/build/testrun/pg_stat_statements...
>
Thanks for the info. Peter sent an email on the pgsql-committers list to
say the same thing. I wrote a patch for this, and sent it to the same list.
See
https://www.postgresql.org/message-id/CAECtzeVXyRruRODFXZ67goMOYtieLbLmcUY6P5L_LrEq8b%3DgxQ%40mail.g...
for more details.
--
Guillaume.
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-12-11 20:55 David Rowley <[email protected]>
parent: David Rowley <[email protected]>
2 siblings, 1 reply; 32+ messages in thread
From: David Rowley @ 2024-12-11 20:55 UTC (permalink / raw)
To: Guillaume Lelarge <[email protected]>; +Cc: Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; Michael Christofides <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Thu, 12 Dec 2024 at 00:16, David Rowley <[email protected]> wrote:
> There were a few places that were missing a BUFFERS OFF as indicated
> by the buildfarm. I've fixed a few of these, but there's at least one
> more in pg_stat_statements's level_tracking.sql. I'm currently running
> a -DRELCACHE_FORCE_RELEASE -DCATCACHE_FORCE_RELEASE enabled make
> check-world. It's going to take a while, so I'll hold off doing yet
> another commit to fix these until I see if the level_tracking.sql one
> is the last issue to fix.
That run didn't show any other failures, so I pushed the patch.
David
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2024-12-13 15:59 Michael Christofides <[email protected]>
parent: David Rowley <[email protected]>
0 siblings, 2 replies; 32+ messages in thread
From: Michael Christofides @ 2024-12-13 15:59 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Guillaume Lelarge <[email protected]>; Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
> I've pushed the main patch.
Woohoo! And thank you. I've already seen quite a lot of positivity around
the commit on Twitter[1][2][3].
> I'm not planning on pushing the auto_explain.log_buffers default change
unless there's a bit more discussion about it.
Much like Guillaume, I'd also be in favour of 0002, but it's nowhere near
as important to me. I think most people consider the parameters far more
when setting up auto_explain, so I believe far fewer omit buffers by
mistake. Also, most cloud providers don't ship with auto_explain on, and
the only one I know of that does[4], ships with log_buffers on too. On the
plus side, it would be nice to be consistent. But on the downside, it might
add a little extra overhead for folks who run auto_explain with log_analyze
on, and who opted not to set log_buffers and upgrade without setting it to
off explicitly. I am still in favour of the 0002 patch being applied, to
avoid confusion and maximise the chance people that don't know about
buffers still get them in their plans.
> do we also need to update doc/src/sgml/rules.sgml?
> https://www.postgresql.org/docs/current/rules-materializedviews.html
Good catch. Testing those file_fdw queries locally, I don't see buffers
reported by the Foreign Scan, but I do initially see some Planning buffers
(on first run). The two plans from the queries on the materialized view do
show buffers now though, of course. Since the file_fdw Foreign Scan is not
reporting buffers, I'm wondering if in this one case simply changing "With
EXPLAIN ANALYZE" to "With EXPLAIN (ANALYZE, BUFFERS OFF)" might be the
least confusing solution?
Thanks again all,
Michael
[1]: https://x.com/nori_shinoda/status/1866805465897898319
[2]: https://x.com/samokhvalov/status/1866863468172939457
[3]: https://x.com/mmeent_pg/status/1866796928002044196
[4]:
https://www.pgmustard.com/blog/which-cloud-providers-support-auto-explain
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2025-02-09 12:05 Jelte Fennema-Nio <[email protected]>
parent: Michael Christofides <[email protected]>
1 sibling, 0 replies; 32+ messages in thread
From: Jelte Fennema-Nio @ 2025-02-09 12:05 UTC (permalink / raw)
To: Michael Christofides <[email protected]>; +Cc: David Rowley <[email protected]>; Guillaume Lelarge <[email protected]>; Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Fri, 13 Dec 2024 at 16:59, Michael Christofides
<[email protected]> wrote:
>
> > I've pushed the main patch.
>
> Woohoo! And thank you. I've already seen quite a lot of positivity around the commit on Twitter[1][2][3].
I wanted to highlight this other patch[1] by Atsushi Torikoshi, which
adds "actual filesystem I/O" metrics to EXPLAIN BUFFERS (and thus now
to default ANALYZE). I marked that patch as "ready for committer" just
now. I'm sharing it here for visibility, because it seems like people
interested in BUFFERS as default would probably also be interested in
that patch. And because it's pretty simple, I think it would be a good
candidate to commit for PG18.
[1]: https://commitfest.postgresql.org/52/5526/
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2025-05-29 04:28 Fujii Masao <[email protected]>
parent: Michael Christofides <[email protected]>
1 sibling, 1 reply; 32+ messages in thread
From: Fujii Masao @ 2025-05-29 04:28 UTC (permalink / raw)
To: Michael Christofides <[email protected]>; David Rowley <[email protected]>; +Cc: Guillaume Lelarge <[email protected]>; Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On 2024/12/14 0:59, Michael Christofides wrote:
> > I've pushed the main patch.
>
> Woohoo! And thank you. I've already seen quite a lot of positivity around the commit on Twitter[1][2][3].
>
> > I'm not planning on pushing the auto_explain.log_buffers default change unless there's a bit more discussion about it.
>
> Much like Guillaume, I'd also be in favour of 0002, but it's nowhere near as important to me. I think most people consider the parameters far more when setting up auto_explain, so I believe far fewer omit buffers by mistake. Also, most cloud providers don't ship with auto_explain on, and the only one I know of that does[4], ships with log_buffers on too. On the plus side, it would be nice to be consistent. But on the downside, it might add a little extra overhead for folks who run auto_explain with log_analyze on, and who opted not to set log_buffers and upgrade without setting it to off explicitly. I am still in favour of the 0002 patch being applied, to avoid confusion and maximise the chance people that don't know about buffers still get them in their plans.
I agree with changing the default value of auto_explain.log_buffers to true.
I think that users who know EXPLAIN ANALYZE includes buffers info in
the result by default since v18 would expect the buffer info also to
be included in auto_explain output as long as log_analyze is enabled.
So setting log_buffers to true by default would be less confusing.
As for 0002 patch, should the example log output with log_analyze enabled
also be updated to show the buffer information?
Sorry for reviving this old thread, but while testing the new v18 feature
- buffers being shown by default - I had the same thought about
auto_explain.log_buffers, so I wanted to chime in.
Regards,
--
Fujii Masao
NTT DATA Japan Corporation
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2025-05-30 11:55 David Rowley <[email protected]>
parent: Fujii Masao <[email protected]>
0 siblings, 1 reply; 32+ messages in thread
From: David Rowley @ 2025-05-30 11:55 UTC (permalink / raw)
To: Fujii Masao <[email protected]>; +Cc: Michael Christofides <[email protected]>; Guillaume Lelarge <[email protected]>; Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Thu, 29 May 2025 at 16:28, Fujii Masao <[email protected]> wrote:
> I agree with changing the default value of auto_explain.log_buffers to true.
> I think that users who know EXPLAIN ANALYZE includes buffers info in
> the result by default since v18 would expect the buffer info also to
> be included in auto_explain output as long as log_analyze is enabled.
> So setting log_buffers to true by default would be less confusing.
I feel somewhat responsible to answer here since I committed the
EXPLAIN ANALYZE change. I'm still neutral.
By my count, those who think we should change log_buffers to true are:
Robert [1], Michael Christofides [2], Fujii [3].
I expect arguments against might be related to log space or
performance. I did a quick test to try to get an idea of what we're
looking at. I'm unsure how realistic it would be that someone would
use auto_explain to log this many plans, but there is a bit of a
performance overhead to enabling log_buffers. Full results attached,
but here's the summary after ignoring the first result of each and
taking the average of the remaining 9 runs of each. I deleted the log
file between each test.
log_min_duration = 0, log_buffers = off: 11231 tps
log_min_duration = 0, log_buffers = on: 11106 tps
log_min_duration = -1 (off): 13297 tps
So it is possible to measure it, at least.
David
[1] https://www.postgresql.org/message-id/CA%2BTgmoZO2HtDGHf7K4rmNdfUGuS%3DngsPgwrH%3DJdyF6Okcnzfkw%40ma...
[2] https://www.postgresql.org/message-id/flat/a6206e08-0607-42fe-91fd-e7bd45fa235a%40oss.nttdata.com#47...
[3] https://www.postgresql.org/message-id/a6206e08-0607-42fe-91fd-e7bd45fa235a%40oss.nttdata.com
# alter system set auto_explain.log_analyze = on;
ALTER SYSTEM
# alter system set auto_explain.log_min_duration=0;
ALTER SYSTEM
# alter system set auto_explain.log_timing=off;
ALTER SYSTEM
# select pg_reload_Conf();
pg_reload_conf
----------------
t
(1 row)
$ pgbench -i postgres
$ for i in {1..10}; do pgbench -n -M prepared -T 10 -S postgres | grep tps; done
tps = 11313.865802
tps = 11353.239133
tps = 11278.210905
tps = 11254.050329
tps = 11322.879711
tps = 11204.294425
tps = 11175.276264
tps = 11211.813427
tps = 11229.452802
tps = 11049.732047
# alter system set auto_explain.log_buffers=1;
ALTER SYSTEM
# select pg_reload_conf();
pg_reload_conf
----------------
t
(1 row)
$ for i in {1..10}; do pgbench -n -M prepared -T 10 -S postgres | grep tps; done
tps = 10509.378765
tps = 11077.139873
tps = 11232.809410
tps = 11141.543652
tps = 11003.065838
tps = 11121.254826
tps = 11136.912487
tps = 11039.627884
tps = 11079.197251
tps = 11122.450776
# alter system set auto_explain.log_min_duration=-1;
ALTER SYSTEM
# select pg_reload_conf();
pg_reload_conf
----------------
t
(1 row)
$ for i in {1..10}; do pgbench -n -M prepared -T 10 -S postgres | grep tps; done
tps = 12663.516909
tps = 13456.062885
tps = 13245.672977
tps = 13326.781348
tps = 13501.902909
tps = 13156.941863
tps = 13212.124942
tps = 13372.313995
tps = 13164.414056
tps = 13236.860009
Attachments:
[text/plain] log_buffers_overhead.txt (1.3K, ../../CAApHDvpqjJqXq2eBXzCqeoAyVANnVg7brrkRiQMENBo8tf44_Q@mail.gmail.com/2-log_buffers_overhead.txt)
download | inline:
# alter system set auto_explain.log_analyze = on;
ALTER SYSTEM
# alter system set auto_explain.log_min_duration=0;
ALTER SYSTEM
# alter system set auto_explain.log_timing=off;
ALTER SYSTEM
# select pg_reload_Conf();
pg_reload_conf
----------------
t
(1 row)
$ pgbench -i postgres
$ for i in {1..10}; do pgbench -n -M prepared -T 10 -S postgres | grep tps; done
tps = 11313.865802
tps = 11353.239133
tps = 11278.210905
tps = 11254.050329
tps = 11322.879711
tps = 11204.294425
tps = 11175.276264
tps = 11211.813427
tps = 11229.452802
tps = 11049.732047
# alter system set auto_explain.log_buffers=1;
ALTER SYSTEM
# select pg_reload_conf();
pg_reload_conf
----------------
t
(1 row)
$ for i in {1..10}; do pgbench -n -M prepared -T 10 -S postgres | grep tps; done
tps = 10509.378765
tps = 11077.139873
tps = 11232.809410
tps = 11141.543652
tps = 11003.065838
tps = 11121.254826
tps = 11136.912487
tps = 11039.627884
tps = 11079.197251
tps = 11122.450776
# alter system set auto_explain.log_min_duration=-1;
ALTER SYSTEM
# select pg_reload_conf();
pg_reload_conf
----------------
t
(1 row)
$ for i in {1..10}; do pgbench -n -M prepared -T 10 -S postgres | grep tps; done
tps = 12663.516909
tps = 13456.062885
tps = 13245.672977
tps = 13326.781348
tps = 13501.902909
tps = 13156.941863
tps = 13212.124942
tps = 13372.313995
tps = 13164.414056
tps = 13236.860009
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2025-05-30 12:43 Jelte Fennema-Nio <[email protected]>
parent: David Rowley <[email protected]>
0 siblings, 1 reply; 32+ messages in thread
From: Jelte Fennema-Nio @ 2025-05-30 12:43 UTC (permalink / raw)
To: David Rowley <[email protected]>; Fujii Masao <[email protected]>; +Cc: Michael Christofides <[email protected]>; Guillaume Lelarge <[email protected]>; Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On Fri May 30, 2025 at 1:55 PM CEST, David Rowley wrote:
> By my count, those who think we should change log_buffers to true are:
> Robert [1], Michael Christofides [2], Fujii [3].
+1 for me on changing the default. It just seems confusing to have
different defaults for auto_explain and EXPLAIN. I'd even vote for
making that change still for PG18, since this feels like an oversight
that the defaults were not kept aligned.
> log_min_duration = 0, log_buffers = off: 11231 tps
> log_min_duration = 0, log_buffers = on: 11106 tps
> log_min_duration = -1 (off): 13297 tps
>
> So it is possible to measure it, at least.
Seems small enough not to worry about it.
^ permalink raw reply [nested|flat] 32+ messages in thread
* Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
@ 2025-05-30 14:36 Fujii Masao <[email protected]>
parent: Jelte Fennema-Nio <[email protected]>
0 siblings, 0 replies; 32+ messages in thread
From: Fujii Masao @ 2025-05-30 14:36 UTC (permalink / raw)
To: Jelte Fennema-Nio <[email protected]>; David Rowley <[email protected]>; +Cc: Michael Christofides <[email protected]>; Guillaume Lelarge <[email protected]>; Greg Sabino Mullane <[email protected]>; Robert Haas <[email protected]>; David G. Johnston <[email protected]>; Nikolay Samokhvalov <[email protected]>; Tom Lane <[email protected]>; pgsql-hackers
On 2025/05/30 21:43, Jelte Fennema-Nio wrote:
> On Fri May 30, 2025 at 1:55 PM CEST, David Rowley wrote:
>> By my count, those who think we should change log_buffers to true are:
>> Robert [1], Michael Christofides [2], Fujii [3].
>
> +1 for me on changing the default. It just seems confusing to have
> different defaults for auto_explain and EXPLAIN. I'd even vote for
> making that change still for PG18, since this feels like an oversight
> that the defaults were not kept aligned.
>
>
>> log_min_duration = 0, log_buffers = off: 11231 tps
>> log_min_duration = 0, log_buffers = on: 11106 tps
>> log_min_duration = -1 (off): 13297 tps
>>
>> So it is possible to measure it, at least.
>
> Seems small enough not to worry about it.
Users can simply disable log_buffers if they enable log_analyze
but don't need buffer usage details and want to avoid the overhead.
If there were no way to avoid the overhead, I'd agree not to make
this change. But since we can disable log_buffers when needed,
I think it's acceptable.
Regards,
--
Fujii Masao
NTT DATA Japan Corporation
^ permalink raw reply [nested|flat] 32+ messages in thread
* [PATCH v47 3/9] Add CONCURRENTLY option to REPACK command.
@ 2026-03-11 14:16 Antonin Houska <[email protected]>
0 siblings, 0 replies; 32+ messages in thread
From: Antonin Houska @ 2026-03-11 14:16 UTC (permalink / raw)
The REPACK command copies the relation data into a new file, creates new
indexes and eventually swaps the files. To make sure that the old file does
not change during the copying, the relation is locked in an exclusive mode,
which prevents applications from both reading and writing. (To keep the data
consistent, we'd only need to prevent the applications from writing, but even
reading needs to be blocked before we can swap the files - otherwise some
applications could continue using the old file. Currently, REPACK takes the
simple approach and acquires the exclusive lock in the beginning.
This patch introduces an alternative workflow, which only requires the
exclusive lock when the relation (and index) files are being swapped.
(Supposedly, the swapping should be pretty fast.) On the other hand, when we
copy the data to the new file, we allow applications to read from the relation
and even to write to it.
First, we scan the relation using a "historic snapshot", and insert all the
tuples satisfying this snapshot into the new relation.
Second, logical decoding is used to capture the data changes done by
applications during the copying (i.e. changes not yet committed from the
perspective of the historic snapshot mentioned above), and those are applied
to the new file before we acquire the exclusive lock that we need to swap the
files. (Of course, more data changes can take place while we are waiting for
the lock - these will be applied to the new file after we have acquired the
lock and before we swap the files.)
While the "concurrent data" changes are applied at specific stages (we cannot
do that until the intial copy is finished and indexes are built), a background
worker performs the decoding all the time. This way we minimize the amount of
not-yet-decoded WAL, so that archiving / recycling of WAL segments is not
delayed much. The decoded changes are written to files and passed to the
backed performing REPACK.
Since the logical decoding system, during its startup, waits until all the
transactions which already have XID assigned have finished, there is a risk of
deadlock if a transaction that already changed anything in the database tries
to acquire a conflicting lock on the table REPACK CONCURRENTLY is working
on. As an example, consider transaction running CREATE INDEX command on the
table that is being REPACKed CONCURRENTLY. On the other hand, DML commands
(INSERT, UPDATE, DELETE) are not a problem as their lock does not conflict
with REPACK CONCURRENTLY.
The current approach is that we accept the risk. If we tried to avoid it, it'd
be necessary to unlock the table before the logical decoding is setup and lock
it again afterwards. Such temporary unlocking would imply re-checking if the
table still meets all the requirements for REPACK CONCURRENTLY.
The WAL records produced by running DML commands on the new relation are
intentionally not fed to the logical decoding system. Doing so would introduce
significant overhead, and - as the new relation is never available for logical
replication - it would be useless.
---
doc/src/sgml/monitoring.sgml | 37 +-
doc/src/sgml/mvcc.sgml | 12 +-
doc/src/sgml/ref/repack.sgml | 112 +-
src/Makefile | 1 +
src/backend/access/heap/heapam.c | 25 +-
src/backend/access/heap/heapam_handler.c | 211 +-
src/backend/access/heap/rewriteheap.c | 6 +-
src/backend/catalog/system_views.sql | 19 +-
src/backend/commands/Makefile | 1 +
src/backend/commands/cluster.c | 1825 ++++++++++++++++-
src/backend/commands/matview.c | 1 +
src/backend/commands/meson.build | 1 +
src/backend/commands/repack_worker.c | 546 +++++
src/backend/commands/tablecmds.c | 1 +
src/backend/commands/vacuum.c | 12 +-
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/libpq/pqmq.c | 5 +
src/backend/meson.build | 1 +
src/backend/postmaster/bgworker.c | 6 +-
src/backend/replication/logical/decode.c | 37 +-
src/backend/replication/logical/logical.c | 6 +-
src/backend/replication/logical/snapbuild.c | 11 +-
.../replication/pgoutput_repack/Makefile | 32 +
.../replication/pgoutput_repack/meson.build | 18 +
.../pgoutput_repack/pgoutput_repack.c | 290 +++
src/backend/replication/walsender.c | 2 +-
src/backend/storage/ipc/procsignal.c | 4 +
.../storage/lmgr/generate-lwlocknames.pl | 2 +-
src/backend/tcop/postgres.c | 4 +
.../utils/activity/wait_event_names.txt | 1 +
src/backend/utils/time/snapmgr.c | 3 +-
src/bin/psql/tab-complete.in.c | 4 +-
src/include/access/heapam_xlog.h | 2 +
src/include/access/tableam.h | 16 +-
src/include/commands/cluster.h | 16 +-
src/include/commands/progress.h | 17 +-
src/include/commands/repack_internal.h | 128 ++
src/include/replication/decode.h | 4 +
src/include/replication/snapbuild.h | 2 +-
src/include/storage/lockdefs.h | 4 +-
src/include/storage/procsignal.h | 1 +
src/include/utils/snapmgr.h | 2 +
src/test/modules/injection_points/Makefile | 2 +
.../injection_points/expected/repack.out | 113 +
.../expected/repack_toast.out | 65 +
src/test/modules/injection_points/meson.build | 2 +
.../injection_points/specs/repack.spec | 142 ++
.../injection_points/specs/repack_toast.spec | 112 +
src/test/regress/expected/rules.out | 19 +-
src/tools/pgindent/typedefs.list | 6 +
50 files changed, 3626 insertions(+), 265 deletions(-)
create mode 100644 src/backend/commands/repack_worker.c
create mode 100644 src/backend/replication/pgoutput_repack/Makefile
create mode 100644 src/backend/replication/pgoutput_repack/meson.build
create mode 100644 src/backend/replication/pgoutput_repack/pgoutput_repack.c
create mode 100644 src/include/commands/repack_internal.h
create mode 100644 src/test/modules/injection_points/expected/repack.out
create mode 100644 src/test/modules/injection_points/expected/repack_toast.out
create mode 100644 src/test/modules/injection_points/specs/repack.spec
create mode 100644 src/test/modules/injection_points/specs/repack_toast.spec
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index bb75ed1069b..a0a155eb10b 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -6985,14 +6985,35 @@ FROM pg_stat_get_backend_idset() AS backendid;
<row>
<entry role="catalog_table_entry"><para role="column_definition">
- <structfield>heap_tuples_written</structfield> <type>bigint</type>
+ <structfield>heap_tuples_inserted</structfield> <type>bigint</type>
</para>
<para>
- Number of heap tuples written.
+ Number of heap tuples inserted.
This counter only advances when the phase is
<literal>seq scanning heap</literal>,
- <literal>index scanning heap</literal>
- or <literal>writing new heap</literal>.
+ <literal>index scanning heap</literal>,
+ <literal>writing new heap</literal>
+ or <literal>catch-up</literal>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>heap_tuples_updated</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of heap tuples updated.
+ This counter only advances when the phase is <literal>catch-up</literal>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>heap_tuples_deleted</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of heap tuples deleted.
+ This counter only advances when the phase is <literal>catch-up</literal>.
</para></entry>
</row>
@@ -7073,6 +7094,14 @@ FROM pg_stat_get_backend_idset() AS backendid;
<command>REPACK</command> is currently writing the new heap.
</entry>
</row>
+ <row>
+ <entry><literal>catch-up</literal></entry>
+ <entry>
+ <command>REPACK CONCURRENTLY</command> is currently processing the DML
+ commands that other transactions executed during any of the preceding
+ phases.
+ </entry>
+ </row>
<row>
<entry><literal>swapping relation files</literal></entry>
<entry>
diff --git a/doc/src/sgml/mvcc.sgml b/doc/src/sgml/mvcc.sgml
index e775260936a..241caeb3593 100644
--- a/doc/src/sgml/mvcc.sgml
+++ b/doc/src/sgml/mvcc.sgml
@@ -1845,15 +1845,17 @@ SELECT pg_advisory_lock(q.id) FROM
<title>Caveats</title>
<para>
- Some DDL commands, currently only <link linkend="sql-truncate"><command>TRUNCATE</command></link> and the
- table-rewriting forms of <link linkend="sql-altertable"><command>ALTER TABLE</command></link>, are not
+ Some commands, currently only <link linkend="sql-truncate"><command>TRUNCATE</command></link>, the
+ table-rewriting forms of <link linkend="sql-altertable"><command>ALTER
+ TABLE</command></link> and <command>REPACK</command> with
+ the <literal>CONCURRENTLY</literal> option, are not
MVCC-safe. This means that after the truncation or rewrite commits, the
table will appear empty to concurrent transactions, if they are using a
- snapshot taken before the DDL command committed. This will only be an
+ snapshot taken before the command committed. This will only be an
issue for a transaction that did not access the table in question
- before the DDL command started — any transaction that has done so
+ before the command started — any transaction that has done so
would hold at least an <literal>ACCESS SHARE</literal> table lock,
- which would block the DDL command until that transaction completes.
+ which would block the truncating or rewriting command until that transaction completes.
So these commands will not cause any apparent inconsistency in the
table contents for successive queries on the target table, but they
could cause visible inconsistency between the contents of the target
diff --git a/doc/src/sgml/ref/repack.sgml b/doc/src/sgml/ref/repack.sgml
index 8ccf7c7a417..170eb84bf43 100644
--- a/doc/src/sgml/ref/repack.sgml
+++ b/doc/src/sgml/ref/repack.sgml
@@ -28,6 +28,7 @@ REPACK [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] USING
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
ANALYZE [ <replaceable class="parameter">boolean</replaceable> ]
+ CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
<phrase>and <replaceable class="parameter">table_and_columns</replaceable> is:</phrase>
@@ -54,7 +55,8 @@ REPACK [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] USING
processes every table and materialized view in the current database that
the current user has the <literal>MAINTAIN</literal> privilege on. This
form of <command>REPACK</command> cannot be executed inside a transaction
- block.
+ block. Also, this form is not allowed if
+ the <literal>CONCURRENTLY</literal> option is used.
</para>
<para>
@@ -67,7 +69,8 @@ REPACK [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] USING
When a table is being repacked, an <literal>ACCESS EXCLUSIVE</literal> lock
is acquired on it. This prevents any other database operations (both reads
and writes) from operating on the table until the <command>REPACK</command>
- is finished.
+ is finished. If you want to keep the table accessible during the repacking,
+ consider using the <literal>CONCURRENTLY</literal> option.
</para>
<refsect2 id="sql-repack-notes-on-clustering" xreflabel="Notes on Clustering">
@@ -198,6 +201,111 @@ REPACK [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] USING
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>CONCURRENTLY</literal></term>
+ <listitem>
+ <para>
+ Allow other transactions to use the table while it is being repacked.
+ </para>
+
+ <para>
+ Internally, <command>REPACK</command> copies the contents of the table
+ (ignoring dead tuples) into a new file, sorted by the specified index,
+ and also creates a new file for each index. Then it swaps the old and
+ new files for the table and all the indexes, and deletes the old
+ files. The <literal>ACCESS EXCLUSIVE</literal> lock is needed to make
+ sure that the old files do not change during the processing because the
+ changes would get lost due to the swap.
+ </para>
+
+ <para>
+ With the <literal>CONCURRENTLY</literal> option, the <literal>ACCESS
+ EXCLUSIVE</literal> lock is only acquired to swap the table and index
+ files. The data changes that took place during the creation of the new
+ table and index files are captured using logical decoding
+ (<xref linkend="logicaldecoding"/>) and applied before
+ the <literal>ACCESS EXCLUSIVE</literal> lock is requested. Thus the lock
+ is typically held only for the time needed to swap the files, which
+ should be pretty short. However, the time might still be noticeable if
+ too many data changes have been done to the table while
+ <command>REPACK</command> was waiting for the lock: those changes must
+ be processed just before the files are swapped, while the
+ <literal>ACCESS EXCLUSIVE</literal> lock is being held.
+ </para>
+
+ <para>
+ Note that <command>REPACK</command> with the
+ <literal>CONCURRENTLY</literal> option does not try to order the rows
+ inserted into the table after the repacking started. Also
+ note <command>REPACK</command> might fail to complete due to DDL
+ commands executed on the table by other transactions during the
+ repacking.
+ </para>
+
+ <note>
+ <para>
+ In addition to the temporary space requirements explained in
+ <xref linkend="sql-repack-notes-on-resources"/>,
+ the <literal>CONCURRENTLY</literal> option can add to the usage of
+ temporary space a bit more. The reason is that other transactions can
+ perform DML operations which cannot be applied to the new file until
+ <command>REPACK</command> has copied all the existing tuples from the
+ old file. Thus the tuples inserted into the old file during the copying
+ are also stored separately in a temporary file, until they can be
+ processed.
+ </para>
+ </note>
+
+ <para>
+ The <literal>CONCURRENTLY</literal> option cannot be used in the
+ following cases:
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The table is <literal>UNLOGGED</literal>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The table is partitioned.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The table is a system catalog or a <acronym>TOAST</acronym> table.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <command>REPACK</command> is executed inside a transaction block.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The <link linkend="guc-max-replication-slots"><varname>max_replication_slots</varname></link>
+ configuration parameter does not allow for creation of an additional
+ replication slot.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+ <warning>
+ <para>
+ <command>REPACK</command> with the <literal>CONCURRENTLY</literal>
+ option is not MVCC-safe, see <xref linkend="mvcc-caveats"/> for
+ details.
+ </para>
+ </warning>
+
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>VERBOSE</literal></term>
<listitem>
diff --git a/src/Makefile b/src/Makefile
index 2f31a2f20a7..b18c9a14ffa 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -23,6 +23,7 @@ SUBDIRS = \
interfaces \
backend/replication/libpqwalreceiver \
backend/replication/pgoutput \
+ backend/replication/pgoutput_repack \
fe_utils \
bin \
pl \
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 0645b2d5f58..6f3dcd73e9d 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -61,7 +61,8 @@ static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tuple,
- bool all_visible_cleared, bool new_all_visible_cleared);
+ bool all_visible_cleared, bool new_all_visible_cleared,
+ bool walLogical);
#ifdef USE_ASSERT_CHECKING
static void check_lock_if_inplace_updateable_rel(Relation relation,
const ItemPointerData *otid,
@@ -2877,6 +2878,7 @@ heap_delete(Relation relation, const ItemPointerData *tid,
uint16 new_infomask,
new_infomask2;
bool changingPart = (options & TABLE_DELETE_CHANGING_PARTITION) != 0;
+ bool walLogical = (options & TABLE_DELETE_NO_LOGICAL) == 0;
bool have_tuple_lock = false;
bool iscombo;
bool all_visible_cleared = false;
@@ -3111,7 +3113,8 @@ l1:
* Compute replica identity tuple before entering the critical section so
* we don't PANIC upon a memory allocation failure.
*/
- old_key_tuple = ExtractReplicaIdentity(relation, &tp, true, &old_key_copied);
+ old_key_tuple = walLogical ?
+ ExtractReplicaIdentity(relation, &tp, true, &old_key_copied) : NULL;
/*
* If this is the first possibly-multixact-able operation in the current
@@ -3201,6 +3204,15 @@ l1:
xlrec.flags |= XLH_DELETE_CONTAINS_OLD_KEY;
}
+ /*
+ * Unlike UPDATE, DELETE is decoded even if there is no old key, so it
+ * does not help to clear both XLH_DELETE_CONTAINS_OLD_TUPLE and
+ * XLH_DELETE_CONTAINS_OLD_KEY. Thus we need an extra flag. TODO
+ * Consider not decoding tuples w/o the old tuple/key instead.
+ */
+ if (!walLogical)
+ xlrec.flags |= XLH_DELETE_NO_LOGICAL;
+
XLogBeginInsert();
XLogRegisterData(&xlrec, SizeOfHeapDelete);
@@ -3351,6 +3363,7 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
HeapTuple heaptup;
HeapTuple old_key_tuple = NULL;
bool old_key_copied = false;
+ bool walLogical = (options & TABLE_UPDATE_NO_LOGICAL) == 0;
Page page,
newpage;
BlockNumber block;
@@ -4232,7 +4245,8 @@ l2:
newbuf, &oldtup, heaptup,
old_key_tuple,
all_visible_cleared,
- all_visible_cleared_new);
+ all_visible_cleared_new,
+ walLogical);
if (newbuf != buffer)
{
PageSetLSN(newpage, recptr);
@@ -8908,7 +8922,8 @@ static XLogRecPtr
log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup, HeapTuple newtup,
HeapTuple old_key_tuple,
- bool all_visible_cleared, bool new_all_visible_cleared)
+ bool all_visible_cleared, bool new_all_visible_cleared,
+ bool walLogical)
{
xl_heap_update xlrec;
xl_heap_header xlhdr;
@@ -8919,7 +8934,7 @@ log_heap_update(Relation reln, Buffer oldbuf,
suffixlen = 0;
XLogRecPtr recptr;
Page page = BufferGetPage(newbuf);
- bool need_tuple_data = RelationIsLogicallyLogged(reln);
+ bool need_tuple_data = walLogical && RelationIsLogicallyLogged(reln);
bool init;
int bufflags;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 69debeff516..df12117236a 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -700,13 +700,14 @@ static void
heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
Relation OldIndex, bool use_sort,
TransactionId OldestXmin,
+ Snapshot snapshot,
TransactionId *xid_cutoff,
MultiXactId *multi_cutoff,
double *num_tuples,
double *tups_vacuumed,
double *tups_recently_dead)
{
- RewriteState rwstate;
+ RewriteState rwstate = NULL;
IndexScanDesc indexScan;
TableScanDesc tableScan;
HeapScanDesc heapScan;
@@ -720,6 +721,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
bool *isnull;
BufferHeapTupleTableSlot *hslot;
BlockNumber prev_cblock = InvalidBlockNumber;
+ bool concurrent = snapshot != NULL;
/* Remember if it's a system catalog */
is_system_catalog = IsSystemRelation(OldHeap);
@@ -735,9 +737,12 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
values = palloc_array(Datum, natts);
isnull = palloc_array(bool, natts);
- /* Initialize the rewrite operation */
- rwstate = begin_heap_rewrite(OldHeap, NewHeap, OldestXmin, *xid_cutoff,
- *multi_cutoff);
+ /*
+ * Initialize the rewrite operation.
+ */
+ if (!concurrent)
+ rwstate = begin_heap_rewrite(OldHeap, NewHeap, OldestXmin,
+ *xid_cutoff, *multi_cutoff);
/* Set up sorting if wanted */
@@ -752,6 +757,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
* Prepare to scan the OldHeap. To ensure we see recently-dead tuples
* that still need to be copied, we scan with SnapshotAny and use
* HeapTupleSatisfiesVacuum for the visibility test.
+ *
+ * In the CONCURRENTLY case, we do regular MVCC visibility tests, using
+ * the snapshot passed by the caller.
*/
if (OldIndex != NULL && !use_sort)
{
@@ -768,7 +776,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
tableScan = NULL;
heapScan = NULL;
- indexScan = index_beginscan(OldHeap, OldIndex, SnapshotAny, NULL, 0, 0,
+ indexScan = index_beginscan(OldHeap, OldIndex,
+ snapshot ? snapshot : SnapshotAny,
+ NULL, 0, 0,
SO_NONE);
index_rescan(indexScan, NULL, 0, NULL, 0);
}
@@ -778,7 +788,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_SEQ_SCAN_HEAP);
- tableScan = table_beginscan(OldHeap, SnapshotAny, 0, (ScanKey) NULL,
+ tableScan = table_beginscan(OldHeap,
+ snapshot ? snapshot : SnapshotAny,
+ 0, (ScanKey) NULL,
SO_NONE);
heapScan = (HeapScanDesc) tableScan;
indexScan = NULL;
@@ -855,83 +867,91 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
buf = hslot->buffer;
/*
- * To be able to guarantee that we can set the hint bit, acquire an
- * exclusive lock on the old buffer. We need the hint bits, set in
- * heapam_relation_copy_for_cluster() -> HeapTupleSatisfiesVacuum(),
- * to be set, as otherwise reform_and_rewrite_tuple() ->
- * rewrite_heap_tuple() will get confused. Specifically,
- * rewrite_heap_tuple() checks for HEAP_XMAX_INVALID in the old tuple
- * to determine whether to check the old-to-new mapping hash table.
- *
- * It'd be better if we somehow could avoid setting hint bits on the
- * old page. One reason to use VACUUM FULL are very bloated tables -
- * rewriting most of the old table during VACUUM FULL doesn't exactly
- * help...
+ * Regarding CONCURRENTLY, see the comments on MVCC snapshot above.
*/
- LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
-
- switch (HeapTupleSatisfiesVacuum(tuple, OldestXmin, buf))
+ if (!concurrent)
{
- case HEAPTUPLE_DEAD:
- /* Definitely dead */
- isdead = true;
- break;
- case HEAPTUPLE_RECENTLY_DEAD:
- *tups_recently_dead += 1;
- pg_fallthrough;
- case HEAPTUPLE_LIVE:
- /* Live or recently dead, must copy it */
- isdead = false;
- break;
- case HEAPTUPLE_INSERT_IN_PROGRESS:
+ /*
+ * To be able to guarantee that we can set the hint bit, acquire
+ * an exclusive lock on the old buffer. We need the hint bits, set
+ * in heapam_relation_copy_for_cluster() ->
+ * HeapTupleSatisfiesVacuum(), to be set, as otherwise
+ * reform_and_rewrite_tuple() -> rewrite_heap_tuple() will get
+ * confused. Specifically, rewrite_heap_tuple() checks for
+ * HEAP_XMAX_INVALID in the old tuple to determine whether to
+ * check the old-to-new mapping hash table.
+ *
+ * It'd be better if we somehow could avoid setting hint bits on
+ * the old page. One reason to use VACUUM FULL are very bloated
+ * tables - rewriting most of the old table during VACUUM FULL
+ * doesn't exactly help...
+ */
+ LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
- /*
- * Since we hold exclusive lock on the relation, normally the
- * only way to see this is if it was inserted earlier in our
- * own transaction. However, it can happen in system
- * catalogs, since we tend to release write lock before commit
- * there. Give a warning if neither case applies; but in any
- * case we had better copy it.
- */
- if (!is_system_catalog &&
- !TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetXmin(tuple->t_data)))
- elog(WARNING, "concurrent insert in progress within table \"%s\"",
- RelationGetRelationName(OldHeap));
- /* treat as live */
- isdead = false;
- break;
- case HEAPTUPLE_DELETE_IN_PROGRESS:
-
- /*
- * Similar situation to INSERT_IN_PROGRESS case.
- */
- if (!is_system_catalog &&
- !TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetUpdateXid(tuple->t_data)))
- elog(WARNING, "concurrent delete in progress within table \"%s\"",
- RelationGetRelationName(OldHeap));
- /* treat as recently dead */
- *tups_recently_dead += 1;
- isdead = false;
- break;
- default:
- elog(ERROR, "unexpected HeapTupleSatisfiesVacuum result");
- isdead = false; /* keep compiler quiet */
- break;
- }
-
- LockBuffer(buf, BUFFER_LOCK_UNLOCK);
-
- if (isdead)
- {
- *tups_vacuumed += 1;
- /* heap rewrite module still needs to see it... */
- if (rewrite_heap_dead_tuple(rwstate, tuple))
+ switch (HeapTupleSatisfiesVacuum(tuple, OldestXmin, buf))
{
- /* A previous recently-dead tuple is now known dead */
- *tups_vacuumed += 1;
- *tups_recently_dead -= 1;
+ case HEAPTUPLE_DEAD:
+ /* Definitely dead */
+ isdead = true;
+ break;
+ case HEAPTUPLE_RECENTLY_DEAD:
+ *tups_recently_dead += 1;
+ pg_fallthrough;
+ case HEAPTUPLE_LIVE:
+ /* Live or recently dead, must copy it */
+ isdead = false;
+ break;
+ case HEAPTUPLE_INSERT_IN_PROGRESS:
+
+ /*
+ * As long as we hold exclusive lock on the relation,
+ * normally the only way to see this is if it was inserted
+ * earlier in our own transaction. However, it can happen
+ * in system catalogs, since we tend to release write lock
+ * before commit there. Give a warning if neither case
+ * applies; but in any case we had better copy it.
+ */
+ if (!is_system_catalog &&
+ !TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetXmin(tuple->t_data)))
+ elog(WARNING, "concurrent insert in progress within table \"%s\"",
+ RelationGetRelationName(OldHeap));
+ /* treat as live */
+ isdead = false;
+ break;
+ case HEAPTUPLE_DELETE_IN_PROGRESS:
+
+ /*
+ * Similar situation to INSERT_IN_PROGRESS case.
+ */
+ if (!is_system_catalog &&
+ !TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetUpdateXid(tuple->t_data)))
+ elog(WARNING, "concurrent delete in progress within table \"%s\"",
+ RelationGetRelationName(OldHeap));
+ /* treat as recently dead */
+ *tups_recently_dead += 1;
+ isdead = false;
+ break;
+ default:
+ elog(ERROR, "unexpected HeapTupleSatisfiesVacuum result");
+ isdead = false; /* keep compiler quiet */
+ break;
+ }
+
+ LockBuffer(buf, BUFFER_LOCK_UNLOCK);
+
+ if (isdead)
+ {
+ *tups_vacuumed += 1;
+ /* heap rewrite module still needs to see it... */
+ if (rewrite_heap_dead_tuple(rwstate, tuple))
+ {
+ /* A previous recently-dead tuple is now known dead */
+ *tups_vacuumed += 1;
+ *tups_recently_dead -= 1;
+ }
+
+ continue;
}
- continue;
}
*num_tuples += 1;
@@ -950,7 +970,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
{
const int ct_index[] = {
PROGRESS_REPACK_HEAP_TUPLES_SCANNED,
- PROGRESS_REPACK_HEAP_TUPLES_WRITTEN
+ PROGRESS_REPACK_HEAP_TUPLES_INSERTED
};
int64 ct_val[2];
@@ -1008,7 +1028,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
values, isnull,
rwstate);
/* Report n_tuples */
- pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_WRITTEN,
+ pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
n_tuples);
}
@@ -1016,7 +1036,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
}
/* Write out any remaining tuples, and fsync if needed */
- end_heap_rewrite(rwstate);
+ if (rwstate)
+ end_heap_rewrite(rwstate);
/* Clean up */
pfree(values);
@@ -2409,6 +2430,10 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
* SET WITHOUT OIDS.
*
* So, we must reconstruct the tuple from component Datums.
+ *
+ * If rwstate=NULL, use simple_heap_insert() instead of rewriting - in that
+ * case we still need to deform/form the tuple. TODO Shouldn't we rename the
+ * function, as might not do any rewrite?
*/
static void
reform_and_rewrite_tuple(HeapTuple tuple,
@@ -2431,8 +2456,28 @@ reform_and_rewrite_tuple(HeapTuple tuple,
copiedTuple = heap_form_tuple(newTupDesc, values, isnull);
- /* The heap rewrite module does the rest */
- rewrite_heap_tuple(rwstate, tuple, copiedTuple);
+ if (rwstate)
+ /* The heap rewrite module does the rest */
+ rewrite_heap_tuple(rwstate, tuple, copiedTuple);
+ else
+ {
+ /*
+ * Insert tuple when processing REPACK CONCURRENTLY.
+ *
+ * rewriteheap.c is not used in the CONCURRENTLY case because it'd be
+ * difficult to do the same in the catch-up phase (as the logical
+ * decoding does not provide us with sufficient visibility
+ * information). Thus we must use heap_insert() both during the
+ * catch-up and here.
+ *
+ * The following is like simple_heap_insert() except that we pass the
+ * flag to skip logical decoding: as soon as REPACK CONCURRENTLY swaps
+ * the relation files, it drops this relation, so no logical
+ * replication subscription should need the data.
+ */
+ heap_insert(NewHeap, copiedTuple, GetCurrentCommandId(true),
+ HEAP_INSERT_NO_LOGICAL, NULL);
+ }
heap_freetuple(copiedTuple);
}
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index 6b19ac3030d..d706856e7a5 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -621,9 +621,9 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
int options = HEAP_INSERT_SKIP_FSM;
/*
- * While rewriting the heap for VACUUM FULL / CLUSTER, make sure data
- * for the TOAST table are not logically decoded. The main heap is
- * WAL-logged as XLOG FPI records, which are not logically decoded.
+ * While rewriting the heap for REPACK, make sure data for the TOAST
+ * table are not logically decoded. The main heap is WAL-logged as
+ * XLOG FPI records, which are not logically decoded.
*/
options |= HEAP_INSERT_NO_LOGICAL;
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index e54018004db..75f97cb23e3 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1343,16 +1343,19 @@ CREATE VIEW pg_stat_progress_repack AS
WHEN 2 THEN 'index scanning heap'
WHEN 3 THEN 'sorting tuples'
WHEN 4 THEN 'writing new heap'
- WHEN 5 THEN 'swapping relation files'
- WHEN 6 THEN 'rebuilding index'
- WHEN 7 THEN 'performing final cleanup'
+ WHEN 5 THEN 'catch-up'
+ WHEN 6 THEN 'swapping relation files'
+ WHEN 7 THEN 'rebuilding index'
+ WHEN 8 THEN 'performing final cleanup'
END AS phase,
CAST(S.param3 AS oid) AS repack_index_relid,
S.param4 AS heap_tuples_scanned,
- S.param5 AS heap_tuples_written,
- S.param6 AS heap_blks_total,
- S.param7 AS heap_blks_scanned,
- S.param8 AS index_rebuild_count
+ S.param5 AS heap_tuples_inserted,
+ S.param6 AS heap_tuples_updated,
+ S.param7 AS heap_tuples_deleted,
+ S.param8 AS heap_blks_total,
+ S.param9 AS heap_blks_scanned,
+ S.param10 AS index_rebuild_count
FROM pg_stat_get_progress_info('REPACK') AS S
LEFT JOIN pg_database D ON S.datid = D.oid;
@@ -1370,7 +1373,7 @@ CREATE VIEW pg_stat_progress_cluster AS
phase,
repack_index_relid AS cluster_index_relid,
heap_tuples_scanned,
- heap_tuples_written,
+ heap_tuples_inserted + heap_tuples_updated AS heap_tuples_written,
heap_blks_total,
heap_blks_scanned,
index_rebuild_count
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index c10fdba2bbb..6926bc44818 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -51,6 +51,7 @@ OBJS = \
proclang.o \
propgraphcmds.o \
publicationcmds.o \
+ repack_worker.o \
schemacmds.o \
seclabel.o \
sequence.o \
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 09066db0956..fd0bfacaae6 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -4,6 +4,22 @@
* REPACK a table; formerly known as CLUSTER. VACUUM FULL also uses
* parts of this code.
*
+ * There are two somewhat different ways to rewrite a table. In non-
+ * concurrent mode, it's easy: take AccessExclusiveLock, create a new
+ * transient relation, copy the tuples over to the relfilenode of the new
+ * relation, swap the relfilenodes, then drop the old relation.
+ *
+ * In concurrent mode, we lock the table with only ShareUpdateExclusiveLock,
+ * then do an initial copy as above. However, while the tuples are being
+ * copied, concurrent transactions could modify the table. To cope with those
+ * changes, we rely on logical decoding to obtain them from WAL. A bgworker
+ * consumes WAL while the initial copy is ongoing (to prevent excessive WAL
+ * from being reserved), and accumulates the changes in a file. Once the
+ * initial copy is complete, we read the changes from the file and re-apply
+ * them on the new heap. Then we upgrade our ShareUpdateExclusiveLock to
+ * AccessExclusiveLock and swap the relfilenodes. This way, the time we hold
+ * a strong lock on the table is much reduced, and the bloat is eliminated.
+ *
*
* Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group
* Portions Copyright (c) 1994-5, Regents of the University of California
@@ -17,6 +33,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/relscan.h"
@@ -24,6 +41,7 @@
#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "access/xloginsert.h"
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
@@ -31,22 +49,29 @@
#include "catalog/namespace.h"
#include "catalog/objectaccess.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_control.h"
#include "catalog/pg_inherits.h"
#include "catalog/toasting.h"
#include "commands/cluster.h"
#include "commands/defrem.h"
#include "commands/progress.h"
+#include "commands/repack_internal.h"
#include "commands/tablecmds.h"
#include "commands/vacuum.h"
+#include "executor/executor.h"
+#include "libpq/pqformat.h"
+#include "libpq/pqmq.h"
#include "miscadmin.h"
#include "optimizer/optimizer.h"
#include "pgstat.h"
#include "storage/bufmgr.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
+#include "storage/proc.h"
#include "utils/acl.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
+#include "utils/injection_point.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
@@ -54,6 +79,7 @@
#include "utils/relmapper.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
+#include "utils/wait_event_types.h"
/*
* This struct is used to pass around the information on tables to be
@@ -66,12 +92,79 @@ typedef struct
Oid indexOid;
} RelToCluster;
+/*
+ * The first file exported by the decoding worker must contain a snapshot, the
+ * following ones contain the data changes.
+ */
+#define WORKER_FILE_SNAPSHOT 0
+
+/*
+ * Information needed to apply concurrent data changes.
+ */
+typedef struct ChangeContext
+{
+ /* The relation the changes are applied to. */
+ Relation cc_rel;
+
+ /* Needed to update indexes of rel_dst. */
+ ResultRelInfo *cc_rri;
+ EState *cc_estate;
+
+ /*
+ * Existing tuples to UPDATE and DELETE are located via this index. We
+ * keep the scankey in partially initialized state to avoid repeated work.
+ * sk_argument is completed on the fly.
+ */
+ Relation cc_ident_index;
+ ScanKey cc_ident_key;
+ int cc_ident_key_nentries;
+
+ /* Sequential number of the file containing the changes. */
+ int cc_file_seq;
+} ChangeContext;
+
+/* FIXME probably in repack_worker.h */
+pid_t backend_pid;
+ProcNumber backend_proc_number;
+
+/*
+ * Backend-local information to control the decoding worker.
+ */
+typedef struct DecodingWorker
+{
+ /* The worker. */
+ BackgroundWorkerHandle *handle;
+
+ /* DecodingWorkerShared is in this segment. */
+ dsm_segment *seg;
+
+ /* Handle of the error queue. */
+ shm_mq_handle *error_mqh;
+} DecodingWorker;
+
+/* Pointer to currently running decoding worker. */
+static DecodingWorker *decoding_worker = NULL;
+
+/*
+ * Is there a message sent by a repack worker that the backend needs to
+ * receive?
+ */
+volatile sig_atomic_t RepackMessagePending = false;
+
+static LOCKMODE RepackLockLevel(bool concurrent);
static bool cluster_rel_recheck(RepackCommand cmd, Relation OldHeap,
- Oid indexOid, Oid userid, int options);
-static void rebuild_relation(Relation OldHeap, Relation index, bool verbose);
+ Oid indexOid, Oid userid, LOCKMODE lmode,
+ int options);
+static void check_repack_concurrently_requirements(Relation rel,
+ Oid *ident_idx_p);
+static void rebuild_relation(Relation OldHeap, Relation index, bool verbose,
+ Oid ident_idx);
static void copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex,
- bool verbose, bool *pSwapToastByContent,
- TransactionId *pFreezeXid, MultiXactId *pCutoffMulti);
+ Snapshot snapshot,
+ bool verbose,
+ bool *pSwapToastByContent,
+ TransactionId *pFreezeXid,
+ MultiXactId *pCutoffMulti);
static List *get_tables_to_repack(RepackCommand cmd, bool usingindex,
MemoryContext permcxt);
static List *get_tables_to_repack_partitioned(RepackCommand cmd,
@@ -79,10 +172,45 @@ static List *get_tables_to_repack_partitioned(RepackCommand cmd,
MemoryContext permcxt);
static bool repack_is_permitted_for_relation(RepackCommand cmd,
Oid relid, Oid userid);
+
+static void apply_concurrent_changes(BufFile *file, ChangeContext *chgcxt);
+static void apply_concurrent_insert(Relation rel, TupleTableSlot *slot,
+ ChangeContext *chgcxt);
+static void apply_concurrent_update(Relation rel, TupleTableSlot *spilled_tuple,
+ TupleTableSlot *ondisk_tuple,
+ ChangeContext *chgcxt);
+static void apply_concurrent_delete(Relation rel, TupleTableSlot *slot);
+static void restore_tuple(BufFile *file, Relation relation,
+ TupleTableSlot *slot);
+static void adjust_toast_pointers(Relation relation, TupleTableSlot *dest,
+ TupleTableSlot *src);
+static bool find_target_tuple(Relation rel, ChangeContext *chgcxt,
+ TupleTableSlot *locator,
+ TupleTableSlot *received);
+static void process_concurrent_changes(XLogRecPtr end_of_wal,
+ ChangeContext *chgcxt,
+ bool done);
+static void initialize_change_context(ChangeContext *chgcxt,
+ Relation relation,
+ Oid ident_index_id);
+static void release_change_context(ChangeContext *chgcxt);
+static void rebuild_relation_finish_concurrent(Relation NewHeap, Relation OldHeap,
+ Oid identIdx,
+ TransactionId frozenXid,
+ MultiXactId cutoffMulti);
+static List *build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes);
static Relation process_single_relation(RepackStmt *stmt,
+ LOCKMODE lockmode,
+ bool isTopLevel,
ClusterParams *params);
static Oid determine_clustered_index(Relation rel, bool usingindex,
const char *indexname);
+
+static void start_repack_decoding_worker(Oid relid);
+static void stop_repack_decoding_worker(void);
+static Snapshot get_initial_snapshot(DecodingWorker *worker);
+
+static void ProcessRepackMessage(StringInfo msg);
static const char *RepackCommandAsString(RepackCommand cmd);
@@ -115,6 +243,7 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
ClusterParams params = {0};
Relation rel = NULL;
MemoryContext repack_context;
+ LOCKMODE lockmode;
List *rtcs;
/* Parse option list */
@@ -125,6 +254,16 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
else if (strcmp(opt->defname, "analyze") == 0 ||
strcmp(opt->defname, "analyse") == 0)
params.options |= defGetBoolean(opt) ? CLUOPT_ANALYZE : 0;
+ else if (strcmp(opt->defname, "concurrently") == 0 &&
+ defGetBoolean(opt))
+ {
+ if (stmt->command != REPACK_COMMAND_REPACK)
+ ereport(ERROR,
+ errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("CONCURRENTLY option not supported for %s",
+ RepackCommandAsString(stmt->command)));
+ params.options |= CLUOPT_CONCURRENT;
+ }
else
ereport(ERROR,
errcode(ERRCODE_SYNTAX_ERROR),
@@ -134,13 +273,16 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
parser_errposition(pstate, opt->location));
}
+ /* Determine the lock mode to use. */
+ lockmode = RepackLockLevel((params.options & CLUOPT_CONCURRENT) != 0);
+
/*
* If a single relation is specified, process it and we're done ... unless
* the relation is a partitioned table, in which case we fall through.
*/
if (stmt->relation != NULL)
{
- rel = process_single_relation(stmt, ¶ms);
+ rel = process_single_relation(stmt, lockmode, isTopLevel, ¶ms);
if (rel == NULL)
return; /* all done */
}
@@ -156,10 +298,29 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
"REPACK (ANALYZE)"));
/*
- * By here, we know we are in a multi-table situation. In order to avoid
- * holding locks for too long, we want to process each table in its own
- * transaction. This forces us to disallow running inside a user
- * transaction block.
+ * By here, we know we are in a multi-table situation.
+ *
+ * Concurrent processing is currently considered rather special (e.g. in
+ * terms of resources consumed) so it is not performed in bulk.
+ */
+ if (params.options & CLUOPT_CONCURRENT)
+ {
+ if (rel != NULL)
+ {
+ Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
+ ereport(ERROR,
+ errmsg("REPACK CONCURRENTLY not supported for partitioned tables"),
+ errhint("Consider running the command for individual partitions."));
+ }
+ else
+ ereport(ERROR,
+ errmsg("REPACK CONCURRENTLY requires explicit table name"));
+ }
+
+ /*
+ * In order to avoid holding locks for too long, we want to process each
+ * table in its own transaction. This forces us to disallow running
+ * inside a user transaction block.
*/
PreventInTransactionBlock(isTopLevel, RepackCommandAsString(stmt->command));
@@ -168,6 +329,12 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
"Repack",
ALLOCSET_DEFAULT_SIZES);
+ /*
+ * Since we open a new transaction for each relation, we have to check
+ * that the relation still is what we think it is.
+ *
+ * In single-transaction CLUSTER, we don't need the overhead.
+ */
params.options |= CLUOPT_RECHECK;
/*
@@ -253,7 +420,7 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
* Open the target table, coping with the case where it has been
* dropped.
*/
- rel = try_table_open(rtc->tableOid, AccessExclusiveLock);
+ rel = try_table_open(rtc->tableOid, lockmode);
if (rel == NULL)
{
CommitTransactionCommand();
@@ -264,7 +431,7 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
PushActiveSnapshot(GetTransactionSnapshot());
/* Process this table */
- cluster_rel(stmt->command, rel, rtc->indexOid, ¶ms);
+ cluster_rel(stmt->command, rel, rtc->indexOid, ¶ms, isTopLevel);
/* cluster_rel closes the relation, but keeps lock */
PopActiveSnapshot();
@@ -278,6 +445,22 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
MemoryContextDelete(repack_context);
}
+/*
+ * In the non-concurrent case, we obtain AccessExclusiveLock throughout the
+ * operation to avoid any lock-upgrade hazards. In the concurrent case, we
+ * grab ShareUpdateExclusiveLock (jsut like VACUUM) for most of the
+ * processing and only acquire AccessExclusiveLock at the end, to swap the
+ * relation -- supposedly for a short time.
+ */
+static LOCKMODE
+RepackLockLevel(bool concurrent)
+{
+ if (concurrent)
+ return ShareUpdateExclusiveLock;
+ else
+ return AccessExclusiveLock;
+}
+
/*
* cluster_rel
*
@@ -293,22 +476,51 @@ ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel)
* If indexOid is InvalidOid, the table will be rewritten in physical order
* instead of index order.
*
+ * Note that, in the concurrent case, the function releases the lock at some
+ * point, in order to get AccessExclusiveLock for the final steps (i.e. to
+ * swap the relation files). To make things simpler, the caller should expect
+ * OldHeap to be closed on return, regardless CLUOPT_CONCURRENT. (The
+ * AccessExclusiveLock is kept till the end of the transaction.)
+ *
* 'cmd' indicates which command is being executed, to be used for error
* messages.
*/
void
cluster_rel(RepackCommand cmd, Relation OldHeap, Oid indexOid,
- ClusterParams *params)
+ ClusterParams *params, bool isTopLevel)
{
Oid tableOid = RelationGetRelid(OldHeap);
+ Relation index;
+ LOCKMODE lmode;
Oid save_userid;
int save_sec_context;
int save_nestlevel;
bool verbose = ((params->options & CLUOPT_VERBOSE) != 0);
bool recheck = ((params->options & CLUOPT_RECHECK) != 0);
- Relation index;
+ bool concurrent = ((params->options & CLUOPT_CONCURRENT) != 0);
+ Oid ident_idx = InvalidOid;
- Assert(CheckRelationLockedByMe(OldHeap, AccessExclusiveLock, false));
+ /* Determine the lock mode to use. */
+ lmode = RepackLockLevel(concurrent);
+
+ /*
+ * Check some preconditions in the concurrent case. This also obtains the
+ * replica index OID.
+ */
+ if (concurrent)
+ {
+ /*
+ * Make sure we're not in a transaction block.
+ *
+ * The reason is that repack_setup_logical_decoding() could deadlock
+ * if there's an XID already assigned. It would be possible to run in
+ * a transaction block if we had no XID, but this restriction is
+ * simpler for users to understand and we don't lose anything.
+ */
+ PreventInTransactionBlock(isTopLevel, "REPACK (CONCURRENTLY)");
+
+ check_repack_concurrently_requirements(OldHeap, &ident_idx);
+ }
/* Check for user-requested abort. */
CHECK_FOR_INTERRUPTS();
@@ -328,16 +540,15 @@ cluster_rel(RepackCommand cmd, Relation OldHeap, Oid indexOid,
RestrictSearchPath();
/*
- * Since we may open a new transaction for each relation, we have to check
- * that the relation still is what we think it is.
+ * Recheck that the relation is still what it was when we started.
*
- * If this is a single-transaction CLUSTER, we can skip these tests. We
- * *must* skip the one on indisclustered since it would reject an attempt
- * to cluster a not-previously-clustered index.
+ * Note that it's critical to skip this in single-relation CLUSTER;
+ * otherwise, we would reject an attempt to cluster using a
+ * not-previously-clustered index.
*/
if (recheck &&
!cluster_rel_recheck(cmd, OldHeap, indexOid, save_userid,
- params->options))
+ lmode, params->options))
goto out;
/*
@@ -353,6 +564,12 @@ cluster_rel(RepackCommand cmd, Relation OldHeap, Oid indexOid,
errmsg("cannot execute %s on a shared catalog",
RepackCommandAsString(cmd)));
+ /*
+ * The CONCURRENTLY case should have been rejected earlier because it does
+ * not support system catalogs.
+ */
+ Assert(!(OldHeap->rd_rel->relisshared && concurrent));
+
/*
* Don't process temp tables of other backends ... their local buffer
* manager is not going to cope.
@@ -374,7 +591,7 @@ cluster_rel(RepackCommand cmd, Relation OldHeap, Oid indexOid,
if (OidIsValid(indexOid))
{
/* verify the index is good and lock it */
- check_index_is_clusterable(OldHeap, indexOid, AccessExclusiveLock);
+ check_index_is_clusterable(OldHeap, indexOid, lmode);
/* also open it */
index = index_open(indexOid, NoLock);
}
@@ -409,7 +626,9 @@ cluster_rel(RepackCommand cmd, Relation OldHeap, Oid indexOid,
if (OldHeap->rd_rel->relkind == RELKIND_MATVIEW &&
!RelationIsPopulated(OldHeap))
{
- relation_close(OldHeap, AccessExclusiveLock);
+ if (index)
+ index_close(index, lmode);
+ relation_close(OldHeap, lmode);
goto out;
}
@@ -422,11 +641,34 @@ cluster_rel(RepackCommand cmd, Relation OldHeap, Oid indexOid,
* invalid, because we move tuples around. Promote them to relation
* locks. Predicate locks on indexes will be promoted when they are
* reindexed.
+ *
+ * During concurrent processing, the heap as well as its indexes stay in
+ * operation, so we postpone this step until they are locked using
+ * AccessExclusiveLock near the end of the processing.
*/
- TransferPredicateLocksToHeapRelation(OldHeap);
+ if (!concurrent)
+ TransferPredicateLocksToHeapRelation(OldHeap);
/* rebuild_relation does all the dirty work */
- rebuild_relation(OldHeap, index, verbose);
+ PG_TRY();
+ {
+ rebuild_relation(OldHeap, index, verbose, ident_idx);
+ }
+ PG_FINALLY();
+ {
+ if (concurrent)
+ {
+ /*
+ * Since during normal operation the worker was already asked to
+ * exit, stopping it explicitly is especially important on ERROR.
+ * However it still seems a good practice to make sure that the
+ * worker never survives the REPACK command.
+ */
+ stop_repack_decoding_worker();
+ }
+ }
+ PG_END_TRY();
+
/* rebuild_relation closes OldHeap, and index if valid */
out:
@@ -445,14 +687,14 @@ out:
*/
static bool
cluster_rel_recheck(RepackCommand cmd, Relation OldHeap, Oid indexOid,
- Oid userid, int options)
+ Oid userid, LOCKMODE lmode, int options)
{
Oid tableOid = RelationGetRelid(OldHeap);
/* Check that the user still has privileges for the relation */
if (!repack_is_permitted_for_relation(cmd, tableOid, userid))
{
- relation_close(OldHeap, AccessExclusiveLock);
+ relation_close(OldHeap, lmode);
return false;
}
@@ -466,7 +708,7 @@ cluster_rel_recheck(RepackCommand cmd, Relation OldHeap, Oid indexOid,
*/
if (RELATION_IS_OTHER_TEMP(OldHeap))
{
- relation_close(OldHeap, AccessExclusiveLock);
+ relation_close(OldHeap, lmode);
return false;
}
@@ -477,7 +719,7 @@ cluster_rel_recheck(RepackCommand cmd, Relation OldHeap, Oid indexOid,
*/
if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(indexOid)))
{
- relation_close(OldHeap, AccessExclusiveLock);
+ relation_close(OldHeap, lmode);
return false;
}
@@ -488,7 +730,7 @@ cluster_rel_recheck(RepackCommand cmd, Relation OldHeap, Oid indexOid,
if ((options & CLUOPT_RECHECK_ISCLUSTERED) != 0 &&
!get_index_isclustered(indexOid))
{
- relation_close(OldHeap, AccessExclusiveLock);
+ relation_close(OldHeap, lmode);
return false;
}
}
@@ -500,7 +742,7 @@ cluster_rel_recheck(RepackCommand cmd, Relation OldHeap, Oid indexOid,
* Verify that the specified heap and index are valid to cluster on
*
* Side effect: obtains lock on the index. The caller may
- * in some cases already have AccessExclusiveLock on the table, but
+ * in some cases already have a lock of the same strength on the table, but
* not in all cases so we can't rely on the table-level lock for
* protection here.
*/
@@ -626,17 +868,94 @@ mark_index_clustered(Relation rel, Oid indexOid, bool is_internal)
}
/*
- * rebuild_relation: rebuild an existing relation in index or physical order
+ * Check if the CONCURRENTLY option is legal for the relation.
*
- * OldHeap: table to rebuild.
- * index: index to cluster by, or NULL to rewrite in physical order.
- *
- * On entry, heap and index (if one is given) must be open, and
- * AccessExclusiveLock held on them.
- * On exit, they are closed, but locks on them are not released.
+ * *Ident_idx_p receives OID of the identity index.
*/
static void
-rebuild_relation(Relation OldHeap, Relation index, bool verbose)
+check_repack_concurrently_requirements(Relation rel, Oid *ident_idx_p)
+{
+ char relpersistence,
+ replident;
+ Oid ident_idx;
+
+ /* Data changes in system relations are not logically decoded. */
+ if (IsCatalogRelation(rel))
+ ereport(ERROR,
+ errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot repack relation \"%s\"",
+ RelationGetRelationName(rel)),
+ errhint("REPACK CONCURRENTLY is not supported for catalog relations."));
+
+ /*
+ * reorderbuffer.c does not seem to handle processing of TOAST relation
+ * alone.
+ */
+ if (IsToastRelation(rel))
+ ereport(ERROR,
+ errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot repack relation \"%s\"",
+ RelationGetRelationName(rel)),
+ errhint("REPACK CONCURRENTLY is not supported for TOAST relations, unless the main relation is repacked too."));
+
+ relpersistence = rel->rd_rel->relpersistence;
+ if (relpersistence != RELPERSISTENCE_PERMANENT)
+ ereport(ERROR,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot repack relation \"%s\"",
+ RelationGetRelationName(rel)),
+ errhint("REPACK CONCURRENTLY is only allowed for permanent relations."));
+
+ /* With NOTHING, WAL does not contain the old tuple. */
+ replident = rel->rd_rel->relreplident;
+ if (replident == REPLICA_IDENTITY_NOTHING)
+ ereport(ERROR,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot repack relation \"%s\"",
+ RelationGetRelationName(rel)),
+ errhint("Relation \"%s\" has insufficient replication identity.",
+ RelationGetRelationName(rel)));
+
+ /*
+ * If the identity index is not set due to replica identity being, PK
+ * might exist.
+ */
+ ident_idx = RelationGetReplicaIndex(rel);
+ if (!OidIsValid(ident_idx) && OidIsValid(rel->rd_pkindex))
+ ident_idx = rel->rd_pkindex;
+ if (!OidIsValid(ident_idx))
+ ereport(ERROR,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot process relation \"%s\"",
+ RelationGetRelationName(rel)),
+ errhint("Relation \"%s\" has no identity index.",
+ RelationGetRelationName(rel)));
+
+ *ident_idx_p = ident_idx;
+}
+
+
+/*
+ * rebuild_relation: rebuild an existing relation in index or physical order
+ *
+ * OldHeap: table to rebuild. See cluster_rel() for comments on the required
+ * lock strength.
+ *
+ * index: index to cluster by, or NULL to rewrite in physical order.
+ *
+ * ident_idx: identity index, to handle replaying of concurrent data changes
+ * to the new heap. InvalidOid if there's no CONCURRENTLY option.
+ *
+ * On entry, heap and index (if one is given) must be open, and the
+ * appropriate lock held on them -- AccessExclusiveLock for exclusive
+ * processing and ShareUpdateExclusiveLock for concurrent processing.
+ *
+ * On exit, they are closed, but still locked with AccessExclusiveLock.
+ * (The function handles the lock upgrade if 'concurrent' is true.)
+ */
+static void
+rebuild_relation(Relation OldHeap, Relation index, bool verbose,
+ Oid ident_idx)
{
Oid tableOid = RelationGetRelid(OldHeap);
Oid accessMethod = OldHeap->rd_rel->relam;
@@ -644,13 +963,55 @@ rebuild_relation(Relation OldHeap, Relation index, bool verbose)
Oid OIDNewHeap;
Relation NewHeap;
char relpersistence;
- bool is_system_catalog;
bool swap_toast_by_content;
TransactionId frozenXid;
MultiXactId cutoffMulti;
+ bool concurrent = OidIsValid(ident_idx);
+ Snapshot snapshot = NULL;
+#if USE_ASSERT_CHECKING
+ LOCKMODE lmode;
- Assert(CheckRelationLockedByMe(OldHeap, AccessExclusiveLock, false) &&
- (index == NULL || CheckRelationLockedByMe(index, AccessExclusiveLock, false)));
+ lmode = concurrent ? ShareUpdateExclusiveLock : AccessExclusiveLock;
+
+ Assert(CheckRelationLockedByMe(OldHeap, lmode, false));
+ Assert(index == NULL || CheckRelationLockedByMe(index, lmode, false));
+#endif
+
+ if (concurrent)
+ {
+ /*
+ * The worker needs to be member of the locking group we're the leader
+ * of. We ought to become the leader before the worker starts. The
+ * worker will join the group as soon as it starts.
+ *
+ * This is to make sure that the deadlock described below is
+ * detectable by deadlock.c: if the worker waits for a transaction to
+ * complete and we are waiting for the worker output, then effectively
+ * we (i.e. this backend) are waiting for that transaction.
+ */
+ BecomeLockGroupLeader();
+
+ /*
+ * Start the worker that decodes data changes applied while we're
+ * copying the table contents.
+ *
+ * Note that the worker has to wait for all transactions with XID
+ * already assigned to finish. If some of those transactions is
+ * waiting for a lock conflicting with ShareUpdateExclusiveLock on our
+ * table (e.g. it runs CREATE INDEX), we can end up in a deadlock.
+ * Not sure this risk is worth unlocking/locking the table (and its
+ * clustering index) and checking again if it's still eligible for
+ * REPACK CONCURRENTLY.
+ */
+ start_repack_decoding_worker(tableOid);
+
+ /*
+ * Wait until the worker has the initial snapshot and retrieve it.
+ */
+ snapshot = get_initial_snapshot(decoding_worker);
+
+ PushActiveSnapshot(snapshot);
+ }
/* for CLUSTER or REPACK USING INDEX, mark the index as the one to use */
if (index != NULL)
@@ -658,7 +1019,6 @@ rebuild_relation(Relation OldHeap, Relation index, bool verbose)
/* Remember info about rel before closing OldHeap */
relpersistence = OldHeap->rd_rel->relpersistence;
- is_system_catalog = IsSystemRelation(OldHeap);
/*
* Create the transient table that will receive the re-ordered data.
@@ -674,30 +1034,59 @@ rebuild_relation(Relation OldHeap, Relation index, bool verbose)
NewHeap = table_open(OIDNewHeap, NoLock);
/* Copy the heap data into the new table in the desired order */
- copy_table_data(NewHeap, OldHeap, index, verbose,
+ copy_table_data(NewHeap, OldHeap, index, snapshot, verbose,
&swap_toast_by_content, &frozenXid, &cutoffMulti);
+ /* The historic snapshot won't be needed anymore. */
+ if (snapshot)
+ {
+ PopActiveSnapshot();
+ UpdateActiveSnapshotCommandId();
+ }
- /* Close relcache entries, but keep lock until transaction commit */
- table_close(OldHeap, NoLock);
- if (index)
- index_close(index, NoLock);
+ if (concurrent)
+ {
+ Assert(!swap_toast_by_content);
- /*
- * Close the new relation so it can be dropped as soon as the storage is
- * swapped. The relation is not visible to others, so no need to unlock it
- * explicitly.
- */
- table_close(NewHeap, NoLock);
+ /*
+ * Close the index, but keep the lock. Both heaps will be closed by
+ * the following call.
+ */
+ if (index)
+ index_close(index, NoLock);
- /*
- * Swap the physical files of the target and transient tables, then
- * rebuild the target's indexes and throw away the transient table.
- */
- finish_heap_swap(tableOid, OIDNewHeap, is_system_catalog,
- swap_toast_by_content, false, true,
- frozenXid, cutoffMulti,
- relpersistence);
+ rebuild_relation_finish_concurrent(NewHeap, OldHeap, ident_idx,
+ frozenXid, cutoffMulti);
+
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ }
+ else
+ {
+ bool is_system_catalog = IsSystemRelation(OldHeap);
+
+ /* Close relcache entries, but keep lock until transaction commit */
+ table_close(OldHeap, NoLock);
+ if (index)
+ index_close(index, NoLock);
+
+ /*
+ * Close the new relation so it can be dropped as soon as the storage
+ * is swapped. The relation is not visible to others, so no need to
+ * unlock it explicitly.
+ */
+ table_close(NewHeap, NoLock);
+
+ /*
+ * Swap the physical files of the target and transient tables, then
+ * rebuild the target's indexes and throw away the transient table.
+ */
+ finish_heap_swap(tableOid, OIDNewHeap, is_system_catalog,
+ swap_toast_by_content, false, true,
+ true, /* reindex */
+ frozenXid, cutoffMulti,
+ relpersistence);
+ }
}
@@ -832,15 +1221,18 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod,
/*
* Do the physical copying of table data.
*
+ * 'snapshot' and 'decoding_ctx': see table_relation_copy_for_cluster(). Pass
+ * iff concurrent processing is required.
+ *
* There are three output parameters:
* *pSwapToastByContent is set true if toast tables must be swapped by content.
* *pFreezeXid receives the TransactionId used as freeze cutoff point.
* *pCutoffMulti receives the MultiXactId used as a cutoff point.
*/
static void
-copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex, bool verbose,
- bool *pSwapToastByContent, TransactionId *pFreezeXid,
- MultiXactId *pCutoffMulti)
+copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex,
+ Snapshot snapshot, bool verbose, bool *pSwapToastByContent,
+ TransactionId *pFreezeXid, MultiXactId *pCutoffMulti)
{
Relation relRelation;
HeapTuple reltup;
@@ -857,6 +1249,10 @@ copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex, bool verb
int elevel = verbose ? INFO : DEBUG2;
PGRUsage ru0;
char *nspname;
+ bool concurrent = snapshot != NULL;
+ LOCKMODE lmode;
+
+ lmode = concurrent ? ShareUpdateExclusiveLock : AccessExclusiveLock;
pg_rusage_init(&ru0);
@@ -885,7 +1281,7 @@ copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex, bool verb
* will be held till end of transaction.
*/
if (OldHeap->rd_rel->reltoastrelid)
- LockRelationOid(OldHeap->rd_rel->reltoastrelid, AccessExclusiveLock);
+ LockRelationOid(OldHeap->rd_rel->reltoastrelid, lmode);
/*
* If both tables have TOAST tables, perform toast swap by content. It is
@@ -894,7 +1290,8 @@ copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex, bool verb
* swap by links. This is okay because swap by content is only essential
* for system catalogs, and we don't support schema changes for them.
*/
- if (OldHeap->rd_rel->reltoastrelid && NewHeap->rd_rel->reltoastrelid)
+ if (OldHeap->rd_rel->reltoastrelid && NewHeap->rd_rel->reltoastrelid &&
+ !concurrent)
{
*pSwapToastByContent = true;
@@ -915,6 +1312,10 @@ copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex, bool verb
* follow the toast pointers to the wrong place. (It would actually
* work for values copied over from the old toast table, but not for
* any values that we toast which were previously not toasted.)
+ *
+ * This would not work with CONCURRENTLY because we may need to delete
+ * TOASTed tuples from the new heap. With this hack, we'd delete them
+ * from the old heap.
*/
NewHeap->rd_toastoid = OldHeap->rd_rel->reltoastrelid;
}
@@ -990,7 +1391,8 @@ copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex, bool verb
* values (e.g. because the AM doesn't use freezing).
*/
table_relation_copy_for_cluster(OldHeap, NewHeap, OldIndex, use_sort,
- cutoffs.OldestXmin, &cutoffs.FreezeLimit,
+ cutoffs.OldestXmin, snapshot,
+ &cutoffs.FreezeLimit,
&cutoffs.MultiXactCutoff,
&num_tuples, &tups_vacuumed,
&tups_recently_dead);
@@ -999,7 +1401,11 @@ copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex, bool verb
*pFreezeXid = cutoffs.FreezeLimit;
*pCutoffMulti = cutoffs.MultiXactCutoff;
- /* Reset rd_toastoid just to be tidy --- it shouldn't be looked at again */
+ /*
+ * Reset rd_toastoid just to be tidy --- it shouldn't be looked at again.
+ * In the CONCURRENTLY case, we need to set it again before applying the
+ * concurrent changes.
+ */
NewHeap->rd_toastoid = InvalidOid;
num_pages = RelationGetNumberOfBlocks(NewHeap);
@@ -1457,14 +1863,13 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
bool swap_toast_by_content,
bool check_constraints,
bool is_internal,
+ bool reindex,
TransactionId frozenXid,
MultiXactId cutoffMulti,
char newrelpersistence)
{
ObjectAddress object;
Oid mapped_tables[4];
- int reindex_flags;
- ReindexParams reindex_params = {0};
int i;
/* Report that we are now swapping relation files */
@@ -1490,39 +1895,47 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
if (is_system_catalog)
CacheInvalidateCatalog(OIDOldHeap);
- /*
- * Rebuild each index on the relation (but not the toast table, which is
- * all-new at this point). It is important to do this before the DROP
- * step because if we are processing a system catalog that will be used
- * during DROP, we want to have its indexes available. There is no
- * advantage to the other order anyway because this is all transactional,
- * so no chance to reclaim disk space before commit. We do not need a
- * final CommandCounterIncrement() because reindex_relation does it.
- *
- * Note: because index_build is called via reindex_relation, it will never
- * set indcheckxmin true for the indexes. This is OK even though in some
- * sense we are building new indexes rather than rebuilding existing ones,
- * because the new heap won't contain any HOT chains at all, let alone
- * broken ones, so it can't be necessary to set indcheckxmin.
- */
- reindex_flags = REINDEX_REL_SUPPRESS_INDEX_USE;
- if (check_constraints)
- reindex_flags |= REINDEX_REL_CHECK_CONSTRAINTS;
+ if (reindex)
+ {
+ int reindex_flags;
+ ReindexParams reindex_params = {0};
- /*
- * Ensure that the indexes have the same persistence as the parent
- * relation.
- */
- if (newrelpersistence == RELPERSISTENCE_UNLOGGED)
- reindex_flags |= REINDEX_REL_FORCE_INDEXES_UNLOGGED;
- else if (newrelpersistence == RELPERSISTENCE_PERMANENT)
- reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
+ /*
+ * Rebuild each index on the relation (but not the toast table, which
+ * is all-new at this point). It is important to do this before the
+ * DROP step because if we are processing a system catalog that will
+ * be used during DROP, we want to have its indexes available. There
+ * is no advantage to the other order anyway because this is all
+ * transactional, so no chance to reclaim disk space before commit. We
+ * do not need a final CommandCounterIncrement() because
+ * reindex_relation does it.
+ *
+ * Note: because index_build is called via reindex_relation, it will
+ * never set indcheckxmin true for the indexes. This is OK even
+ * though in some sense we are building new indexes rather than
+ * rebuilding existing ones, because the new heap won't contain any
+ * HOT chains at all, let alone broken ones, so it can't be necessary
+ * to set indcheckxmin.
+ */
+ reindex_flags = REINDEX_REL_SUPPRESS_INDEX_USE;
+ if (check_constraints)
+ reindex_flags |= REINDEX_REL_CHECK_CONSTRAINTS;
- /* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ /*
+ * Ensure that the indexes have the same persistence as the parent
+ * relation.
+ */
+ if (newrelpersistence == RELPERSISTENCE_UNLOGGED)
+ reindex_flags |= REINDEX_REL_FORCE_INDEXES_UNLOGGED;
+ else if (newrelpersistence == RELPERSISTENCE_PERMANENT)
+ reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
- reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
+ /* Report that we are now reindexing relations */
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+
+ reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
+ }
/* Report that we are now doing clean up */
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
@@ -1566,6 +1979,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
object.objectId = OIDNewHeap;
object.objectSubId = 0;
+ if (!reindex)
+ {
+ /*
+ * Make sure the changes in pg_class are visible. This is especially
+ * important if !swap_toast_by_content, so that the correct TOAST
+ * relation is dropped. (reindex_relation() above did not help in this
+ * case))
+ */
+ CommandCounterIncrement();
+ }
+
/*
* The new relation is local to our transaction and we know nothing
* depends on it, so DROP_RESTRICT should be OK.
@@ -1605,7 +2029,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
/* Get the associated valid index to be renamed */
toastidx = toast_get_valid_index(newrel->rd_rel->reltoastrelid,
- NoLock);
+ AccessExclusiveLock);
/* rename the toast table ... */
snprintf(NewToastName, NAMEDATALEN, "pg_toast_%u",
@@ -1876,7 +2300,8 @@ repack_is_permitted_for_relation(RepackCommand cmd, Oid relid, Oid userid)
* case, if an index name is given, it's up to the caller to resolve it.
*/
static Relation
-process_single_relation(RepackStmt *stmt, ClusterParams *params)
+process_single_relation(RepackStmt *stmt, LOCKMODE lockmode, bool isTopLevel,
+ ClusterParams *params)
{
Relation rel;
Oid tableOid;
@@ -1893,13 +2318,9 @@ process_single_relation(RepackStmt *stmt, ClusterParams *params)
errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("ANALYZE option must be specified when a column list is provided"));
- /*
- * Find, lock, and check permissions on the table. We obtain
- * AccessExclusiveLock right away to avoid lock-upgrade hazard in the
- * single-transaction case.
- */
+ /* Find, lock, and check permissions on the table. */
tableOid = RangeVarGetRelidExtended(stmt->relation->relation,
- AccessExclusiveLock,
+ lockmode,
0,
RangeVarCallbackMaintainsTable,
NULL);
@@ -1924,13 +2345,14 @@ process_single_relation(RepackStmt *stmt, ClusterParams *params)
return rel;
else
{
- Oid indexOid;
+ Oid indexOid = InvalidOid;
indexOid = determine_clustered_index(rel, stmt->usingindex,
stmt->indexname);
if (OidIsValid(indexOid))
- check_index_is_clusterable(rel, indexOid, AccessExclusiveLock);
- cluster_rel(stmt->command, rel, indexOid, params);
+ check_index_is_clusterable(rel, indexOid, lockmode);
+
+ cluster_rel(stmt->command, rel, indexOid, params, isTopLevel);
/*
* Do an analyze, if requested. We close the transaction and start a
@@ -2025,3 +2447,1184 @@ RepackCommandAsString(RepackCommand cmd)
}
return "???"; /* keep compiler quiet */
}
+
+/*
+ * Apply all the changes stored in 'file'.
+ */
+static void
+apply_concurrent_changes(BufFile *file, ChangeContext *chgcxt)
+{
+ ConcurrentChangeKind kind = '\0';
+ Relation rel = chgcxt->cc_rel;
+ TupleTableSlot *spilled_tuple;
+ TupleTableSlot *old_update_tuple;
+ TupleTableSlot *ondisk_tuple;
+ MemoryContext apply_cxt;
+ bool have_old_tuple = false;
+
+ /*
+ * Use a separate memory context for the tuples and any memory needed to
+ * process them.
+ *
+ * XXX would this be better with GenerationContextCreate?
+ */
+ apply_cxt = AllocSetContextCreate(TopTransactionContext,
+ "REPACK - apply",
+ ALLOCSET_DEFAULT_SIZES);
+
+ spilled_tuple = MakeSingleTupleTableSlot(RelationGetDescr(rel),
+ &TTSOpsVirtual);
+ ondisk_tuple = MakeSingleTupleTableSlot(RelationGetDescr(rel),
+ table_slot_callbacks(rel));
+ old_update_tuple = MakeSingleTupleTableSlot(RelationGetDescr(rel),
+ &TTSOpsVirtual);
+
+ while (true)
+ {
+ size_t nread;
+ ConcurrentChangeKind prevkind = kind;
+
+ CHECK_FOR_INTERRUPTS();
+
+ nread = BufFileReadMaybeEOF(file, &kind, 1, true);
+ if (nread == 0) /* done with the file? */
+ break;
+
+ /*
+ * If this is the old tuple for an update, read it into the tuple slot
+ * and go to the next one. The update itself will be executed on the
+ * next iteration, when we receive the NEW tuple.
+ */
+ if (kind == CHANGE_UPDATE_OLD)
+ {
+ restore_tuple(file, rel, old_update_tuple);
+ have_old_tuple = true;
+ continue;
+ }
+
+ /*
+ * Just before an UPDATE or DELETE, we must update the command
+ * counter, because the change could refer to a tuple that we have
+ * just inserted; and before an INSERT, we have to do this also if the
+ * previous command was either update or delete.
+ *
+ * With this approach we don't spend so many CCIs for long strings of
+ * only INSERTs, which can't affect one another.
+ */
+ if (kind == CHANGE_UPDATE_NEW || kind == CHANGE_DELETE ||
+ (kind == CHANGE_INSERT && (prevkind == CHANGE_UPDATE_NEW ||
+ prevkind == CHANGE_DELETE)))
+ {
+ CommandCounterIncrement();
+ UpdateActiveSnapshotCommandId();
+ }
+
+ /*
+ * Now restore the tuple into the slot and execute the change.
+ */
+ restore_tuple(file, rel, spilled_tuple);
+
+ if (kind == CHANGE_INSERT)
+ {
+ apply_concurrent_insert(rel, spilled_tuple, chgcxt);
+ }
+ else if (kind == CHANGE_DELETE)
+ {
+ bool found;
+
+ /* Find the tuple to be deleted */
+ found = find_target_tuple(rel, chgcxt, spilled_tuple, ondisk_tuple);
+ if (!found)
+ elog(ERROR, "failed to find target tuple");
+ apply_concurrent_delete(rel, ondisk_tuple);
+ }
+ else if (kind == CHANGE_UPDATE_NEW)
+ {
+ TupleTableSlot *key;
+ bool found;
+
+ if (have_old_tuple)
+ key = old_update_tuple;
+ else
+ key = spilled_tuple;
+
+ /* Find the tuple to be updated or deleted. */
+ found = find_target_tuple(rel, chgcxt, key, ondisk_tuple);
+ if (!found)
+ elog(ERROR, "failed to find target tuple");
+
+ /*
+ * If 'tup' contains TOAST pointers, they point to the old
+ * relation's toast. Copy the corresponding TOAST pointers for the
+ * new relation from the existing tuple. (The fact that we
+ * received a TOAST pointer here implies that the attribute hasn't
+ * changed.)
+ */
+ adjust_toast_pointers(rel, spilled_tuple, ondisk_tuple);
+
+ apply_concurrent_update(rel, spilled_tuple, ondisk_tuple, chgcxt);
+
+ ExecClearTuple(old_update_tuple);
+ have_old_tuple = false;
+ }
+ else
+ elog(ERROR, "unrecognized kind of change: %d", kind);
+
+ MemoryContextReset(apply_cxt);
+ }
+
+ /* Cleanup. */
+ ExecDropSingleTupleTableSlot(spilled_tuple);
+ ExecDropSingleTupleTableSlot(ondisk_tuple);
+ ExecDropSingleTupleTableSlot(old_update_tuple);
+ MemoryContextDelete(apply_cxt);
+}
+
+/*
+ * Apply an insert from the spill of concurrent changes to the new copy of the
+ * table.
+ */
+static void
+apply_concurrent_insert(Relation rel, TupleTableSlot *slot,
+ ChangeContext *chgcxt)
+{
+ List *recheck;
+
+ /* Put the tuple in the table, but make sure it won't be decoded */
+ table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+ HEAP_INSERT_NO_LOGICAL, NULL);
+
+ /*
+ * Update indexes with this new tuple. Because we're merely replaying an
+ * action that already happened, we have no use for the recheck list of
+ * indexes returned, so just free it. XXX or maybe just leave it?
+ */
+ recheck = ExecInsertIndexTuples(chgcxt->cc_rri,
+ chgcxt->cc_estate,
+ 0,
+ slot,
+ NIL, NULL);
+ list_free(recheck);
+
+ pgstat_progress_incr_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED, 1);
+
+ ResetPerTupleExprContext(chgcxt->cc_estate);
+}
+
+/*
+ * Apply an update from the spill of concurrent changes to the new copy of the
+ * table.
+ */
+static void
+apply_concurrent_update(Relation rel, TupleTableSlot *spilled_tuple,
+ TupleTableSlot *ondisk_tuple,
+ ChangeContext *chgcxt)
+{
+ LockTupleMode lockmode;
+ TM_FailureData tmfd;
+ TU_UpdateIndexes update_indexes;
+ TM_Result res;
+ List *recheck;
+
+ /*
+ * Carry out the update, avoiding logical decoding info.
+ */
+ res = table_tuple_update(rel, &(ondisk_tuple->tts_tid), spilled_tuple,
+ GetCurrentCommandId(true),
+ TABLE_UPDATE_NO_LOGICAL,
+ InvalidSnapshot,
+ InvalidSnapshot,
+ false,
+ &tmfd, &lockmode, &update_indexes);
+ if (res != TM_Ok)
+ ereport(ERROR,
+ errmsg("failed to apply concurrent UPDATE"));
+
+ if (update_indexes != TU_None)
+ {
+ uint32 flags = EIIT_IS_UPDATE;
+
+ if (update_indexes == TU_Summarizing)
+ flags |= EIIT_ONLY_SUMMARIZING;
+ recheck = ExecInsertIndexTuples(chgcxt->cc_rri,
+ chgcxt->cc_estate,
+ flags,
+ spilled_tuple,
+ NIL, NULL);
+ list_free(recheck);
+ ResetPerTupleExprContext(chgcxt->cc_estate);
+ }
+
+ pgstat_progress_incr_param(PROGRESS_REPACK_HEAP_TUPLES_UPDATED, 1);
+}
+
+static void
+apply_concurrent_delete(Relation rel, TupleTableSlot *slot)
+{
+ TM_Result res;
+ TM_FailureData tmfd;
+
+ /*
+ * Delete tuple from the new heap.
+ *
+ * Do it like in simple_heap_delete(), except for 'wal_logical' (and
+ * except for 'wait').
+ */
+ res = table_tuple_delete(rel, &(slot->tts_tid),
+ GetCurrentCommandId(true),
+ TABLE_DELETE_NO_LOGICAL,
+ InvalidSnapshot, InvalidSnapshot,
+ false,
+ &tmfd);
+
+ if (res != TM_Ok)
+ ereport(ERROR,
+ errmsg("failed to apply concurrent DELETE"));
+
+ pgstat_progress_incr_param(PROGRESS_REPACK_HEAP_TUPLES_DELETED, 1);
+}
+
+/*
+ * Read tuple from file and put it in the input slot.
+ *
+ * External attributes are stored in separate memory chunks, in order to avoid
+ * exceeding MaxAllocSize - that could happen if the individual attributes are
+ * smaller than MaxAllocSize but the whole tuple is bigger.
+ */
+static void
+restore_tuple(BufFile *file, Relation relation, TupleTableSlot *slot)
+{
+ uint32 t_len;
+ HeapTuple tup;
+ MemoryContext oldcxt;
+ int natt_ext;
+
+ oldcxt = MemoryContextSwitchTo(slot->tts_mcxt);
+
+ /* Read the tuple. */
+ BufFileReadExact(file, &t_len, sizeof(t_len));
+ tup = (HeapTuple) palloc(HEAPTUPLESIZE + t_len);
+ tup->t_data = (HeapTupleHeader) ((char *) tup + HEAPTUPLESIZE);
+ BufFileReadExact(file, tup->t_data, t_len);
+ tup->t_len = t_len;
+ ItemPointerSetInvalid(&tup->t_self);
+ tup->t_tableOid = RelationGetRelid(relation);
+
+ /*
+ * Put the tuple we read in a slot. This deforms it, so that we can hack
+ * the external attributes in place.
+ */
+ ExecForceStoreHeapTuple(tup, slot, false);
+
+ /*
+ * Next, read any attributes we stored separately into the tts_values
+ * array elements expecting them, if any. This matches
+ * repack_store_change.
+ */
+ BufFileReadExact(file, &natt_ext, sizeof(natt_ext));
+ if (natt_ext > 0)
+ {
+ TupleDesc desc = slot->tts_tupleDescriptor;
+
+ for (int i = 0; i < desc->natts; i++)
+ {
+ CompactAttribute *attr = TupleDescCompactAttr(desc, i);
+ varlena *varlen;
+ union
+ {
+ alignas(int32) varlena hdr;
+ char data[sizeof(void *)];
+ } chunk_header;
+ void *value;
+ Size varlensz;
+
+ if (attr->attisdropped || attr->attlen != -1)
+ continue;
+ if (slot_attisnull(slot, i + 1))
+ continue;
+ varlen = (varlena *) DatumGetPointer(slot->tts_values[i]);
+ if (!VARATT_IS_EXTERNAL(varlen))
+ continue;
+ slot_getsomeattrs(slot, i + 1);
+
+ BufFileReadExact(file, &chunk_header, VARHDRSZ);
+ varlensz = VARSIZE_ANY(&chunk_header);
+
+ value = palloc(varlensz);
+ SET_VARSIZE(value, VARSIZE_ANY(&chunk_header));
+ BufFileReadExact(file, (char *) value + VARHDRSZ, varlensz - VARHDRSZ);
+
+ slot->tts_values[i] = PointerGetDatum(value);
+ natt_ext--;
+ }
+ }
+
+ MemoryContextSwitchTo(oldcxt);
+}
+
+/*
+ * Adjust 'dest' replacing any EXTERNAL_ONDISK toast pointers with the
+ * corresponding ones from 'src'.
+ */
+static void
+adjust_toast_pointers(Relation relation, TupleTableSlot *dest, TupleTableSlot *src)
+{
+ TupleDesc desc = dest->tts_tupleDescriptor;
+
+ for (int i = 0; i < desc->natts; i++)
+ {
+ CompactAttribute *attr = TupleDescCompactAttr(desc, i);
+ varlena *varlena_dst;
+
+ if (attr->attisdropped)
+ continue;
+ if (attr->attlen != -1)
+ continue;
+ if (slot_attisnull(dest, i + 1))
+ continue;
+
+ slot_getsomeattrs(dest, i + 1);
+
+ varlena_dst = (varlena *) DatumGetPointer(dest->tts_values[i]);
+ if (!VARATT_IS_EXTERNAL_ONDISK(varlena_dst))
+ continue;
+ slot_getsomeattrs(src, i + 1);
+
+ /*
+ * XXX We simply replace the pointer to the Datum from the other one,
+ * which is probably bogus.
+ */
+ dest->tts_values[i] = src->tts_values[i];
+ }
+}
+
+/*
+ * Find the tuple to be updated or deleted by the given data change, whose
+ * tuple has already been loaded into locator.
+ *
+ * If the tuple is found, put it in retrieved and return true. If the tuple is
+ * not found, return false.
+ */
+static bool
+find_target_tuple(Relation rel, ChangeContext *chgcxt, TupleTableSlot *locator,
+ TupleTableSlot *retrieved)
+{
+ Form_pg_index idx = chgcxt->cc_ident_index->rd_index;
+ IndexScanDesc scan;
+ bool retval;
+
+ /*
+ * Scan key is passed by caller, so it does not have to be constructed
+ * multiple times. Key entries have all fields initialized, except for
+ * sk_argument.
+ *
+ * Use the incoming tuple to finalize the scan key.
+ */
+ for (int i = 0; i < chgcxt->cc_ident_key_nentries; i++)
+ {
+ ScanKey entry = &chgcxt->cc_ident_key[i];
+ AttrNumber attno = idx->indkey.values[i];
+
+ entry->sk_argument = locator->tts_values[attno - 1];
+ Assert(!locator->tts_isnull[attno - 1]);
+ }
+
+ /* XXX no instrumentation for now */
+ scan = index_beginscan(rel, chgcxt->cc_ident_index, GetActiveSnapshot(),
+ NULL, chgcxt->cc_ident_key_nentries, 0, 0);
+ index_rescan(scan, chgcxt->cc_ident_key, chgcxt->cc_ident_key_nentries, NULL, 0);
+ retval = index_getnext_slot(scan, ForwardScanDirection, retrieved);
+ index_endscan(scan);
+
+ return retval;
+}
+
+/*
+ * Decode and apply concurrent changes, up to (and including) the record whose
+ * LSN is 'end_of_wal'.
+ *
+ * XXX the names "process_concurrent_changes" and "apply_concurrent_changes"
+ * are far too similar to each other.
+ */
+static void
+process_concurrent_changes(XLogRecPtr end_of_wal, ChangeContext *chgcxt, bool done)
+{
+ DecodingWorkerShared *shared;
+ char fname[MAXPGPATH];
+ BufFile *file;
+
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_CATCH_UP);
+
+ /* Ask the worker for the file. */
+ shared = (DecodingWorkerShared *) dsm_segment_address(decoding_worker->seg);
+ SpinLockAcquire(&shared->mutex);
+ shared->lsn_upto = end_of_wal;
+ shared->done = done;
+ SpinLockRelease(&shared->mutex);
+
+ /*
+ * The worker needs to finish processing of the current WAL record. Even
+ * if it's idle, it'll need to close the output file. Thus we're likely to
+ * wait, so prepare for sleep.
+ */
+ ConditionVariablePrepareToSleep(&shared->cv);
+ for (;;)
+ {
+ int last_exported;
+
+ SpinLockAcquire(&shared->mutex);
+ last_exported = shared->last_exported;
+ SpinLockRelease(&shared->mutex);
+
+ /*
+ * Has the worker exported the file we are waiting for?
+ */
+ if (last_exported == chgcxt->cc_file_seq)
+ break;
+
+ ConditionVariableSleep(&shared->cv, WAIT_EVENT_REPACK_WORKER_EXPORT);
+ }
+ ConditionVariableCancelSleep();
+
+ /* Open the file. */
+ DecodingWorkerFileName(fname, shared->relid, chgcxt->cc_file_seq);
+ file = BufFileOpenFileSet(&shared->sfs.fs, fname, O_RDONLY, false);
+ apply_concurrent_changes(file, chgcxt);
+
+ BufFileClose(file);
+
+ /* Get ready for the next file. */
+ chgcxt->cc_file_seq++;
+}
+
+/*
+ * Initialize the ChangeContext struct for the given relation, with
+ * the given index as identity index.
+ */
+static void
+initialize_change_context(ChangeContext *chgcxt,
+ Relation relation, Oid ident_index_id)
+{
+ chgcxt->cc_rel = relation;
+
+ /* Only initialize fields needed by ExecInsertIndexTuples(). */
+ chgcxt->cc_estate = CreateExecutorState();
+
+ chgcxt->cc_rri = (ResultRelInfo *) palloc(sizeof(ResultRelInfo));
+ InitResultRelInfo(chgcxt->cc_rri, relation, 0, 0, 0);
+ ExecOpenIndices(chgcxt->cc_rri, false);
+
+ /*
+ * The table's relcache entry already has the relcache entry for the
+ * identity index; find that.
+ */
+ chgcxt->cc_ident_index = NULL;
+ for (int i = 0; i < chgcxt->cc_rri->ri_NumIndices; i++)
+ {
+ Relation ind_rel;
+
+ ind_rel = chgcxt->cc_rri->ri_IndexRelationDescs[i];
+ if (ind_rel->rd_id == ident_index_id)
+ {
+ chgcxt->cc_ident_index = ind_rel;
+ break;
+ }
+ }
+ if (chgcxt->cc_ident_index == NULL)
+ elog(ERROR, "failed to find identity index");
+
+ /* Set up for scanning said identity index */
+ {
+ Form_pg_index indexForm;
+
+ indexForm = chgcxt->cc_ident_index->rd_index;
+ chgcxt->cc_ident_key_nentries = indexForm->indnkeyatts;
+ chgcxt->cc_ident_key = (ScanKey) palloc_array(ScanKeyData, indexForm->indnkeyatts);
+ for (int i = 0; i < indexForm->indnkeyatts; i++)
+ {
+ ScanKey entry;
+ Oid opfamily,
+ opcintype,
+ opno,
+ opcode;
+
+ entry = &chgcxt->cc_ident_key[i];
+
+ opfamily = chgcxt->cc_ident_index->rd_opfamily[i];
+ opcintype = chgcxt->cc_ident_index->rd_opcintype[i];
+ opno = get_opfamily_member(opfamily, opcintype, opcintype,
+ BTEqualStrategyNumber);
+ if (!OidIsValid(opno))
+ elog(ERROR, "failed to find = operator for type %u", opcintype);
+ opcode = get_opcode(opno);
+ if (!OidIsValid(opcode))
+ elog(ERROR, "failed to find = operator for operator %u", opno);
+
+ /* Initialize everything but argument. */
+ ScanKeyInit(entry,
+ i + 1,
+ BTEqualStrategyNumber, opcode,
+ (Datum) NULL);
+ entry->sk_collation = chgcxt->cc_ident_index->rd_indcollation[i];
+ }
+ }
+
+ chgcxt->cc_file_seq = WORKER_FILE_SNAPSHOT + 1;
+}
+
+/*
+ * Free up resources taken by a ChangeContext.
+ */
+static void
+release_change_context(ChangeContext *chgcxt)
+{
+ ExecCloseIndices(chgcxt->cc_rri);
+ FreeExecutorState(chgcxt->cc_estate);
+ /* XXX are these pfrees necessary? */
+ pfree(chgcxt->cc_rri);
+ pfree(chgcxt->cc_ident_key);
+}
+
+/*
+ * The final steps of rebuild_relation() for concurrent processing.
+ *
+ * On entry, NewHeap is locked in AccessExclusiveLock mode. OldHeap and its
+ * clustering index (if one is passed) are still locked in a mode that allows
+ * concurrent data changes. On exit, both tables and their indexes are closed,
+ * but locked in AccessExclusiveLock mode.
+ */
+static void
+rebuild_relation_finish_concurrent(Relation NewHeap, Relation OldHeap,
+ Oid identIdx, TransactionId frozenXid,
+ MultiXactId cutoffMulti)
+{
+ LOCKMODE lockmode_old PG_USED_FOR_ASSERTS_ONLY;
+ List *ind_oids_new;
+ Oid old_table_oid = RelationGetRelid(OldHeap);
+ Oid new_table_oid = RelationGetRelid(NewHeap);
+ List *ind_oids_old = RelationGetIndexList(OldHeap);
+ ListCell *lc,
+ *lc2;
+ char relpersistence;
+ bool is_system_catalog;
+ Oid ident_idx_new;
+ XLogRecPtr wal_insert_ptr,
+ end_of_wal;
+ char dummy_rec_data = '\0';
+ Relation *ind_refs,
+ *ind_refs_p;
+ int nind;
+ ChangeContext chgcxt;
+
+ /* Like in cluster_rel(). */
+ lockmode_old = ShareUpdateExclusiveLock;
+ Assert(CheckRelationLockedByMe(OldHeap, lockmode_old, false));
+ /* This is expected from the caller. */
+ Assert(CheckRelationLockedByMe(NewHeap, AccessExclusiveLock, false));
+
+ /*
+ * Unlike the exclusive case, we build new indexes for the new relation
+ * rather than swapping the storage and reindexing the old relation. The
+ * point is that the index build can take some time, so we do it before we
+ * get AccessExclusiveLock on the old heap and therefore we cannot swap
+ * the heap storage yet.
+ *
+ * index_create() will lock the new indexes using AccessExclusiveLock - no
+ * need to change that. At the same time, we use ShareUpdateExclusiveLock
+ * to lock the existing indexes - that should be enough to prevent others
+ * from changing them while we're repacking the relation. The lock on
+ * table should prevent others from changing the index column list, but
+ * might not be enough for commands like ALTER INDEX ... SET ... (Those
+ * are not necessarily dangerous, but can make user confused if the
+ * changes they do get lost due to REPACK.)
+ */
+ ind_oids_new = build_new_indexes(NewHeap, OldHeap, ind_oids_old);
+
+ /* Find "identity index" on the new relation. */
+ ident_idx_new = InvalidOid;
+ forboth(lc, ind_oids_old, lc2, ind_oids_new)
+ {
+ Oid ind_old = lfirst_oid(lc);
+ Oid ind_new = lfirst_oid(lc2);
+
+ if (identIdx == ind_old)
+ {
+ ident_idx_new = ind_new;
+ break;
+ }
+ }
+
+ /* Should not happen, given our lock on the old relation. */
+ if (!OidIsValid(ident_idx_new))
+ ereport(ERROR,
+ errmsg("identity index missing on the new relation"));
+
+ /* Gather information to apply concurrent changes. */
+ initialize_change_context(&chgcxt, NewHeap, ident_idx_new);
+
+ /*
+ * During testing, wait for another backend to perform concurrent data
+ * changes which we will process below.
+ */
+ INJECTION_POINT("repack-concurrently-before-lock", NULL);
+
+ /*
+ * Flush all WAL records inserted so far (possibly except for the last
+ * incomplete page; see GetInsertRecPtr), to minimize the amount of data
+ * we need to flush while holding exclusive lock on the source table.
+ */
+ wal_insert_ptr = GetInsertRecPtr();
+ XLogFlush(wal_insert_ptr);
+ end_of_wal = GetFlushRecPtr(NULL);
+
+ /*
+ * Apply concurrent changes first time, to minimize the time we need to
+ * hold AccessExclusiveLock. (Quite some amount of WAL could have been
+ * written during the data copying and index creation.)
+ */
+ process_concurrent_changes(end_of_wal, &chgcxt, false);
+
+ /*
+ * Acquire AccessExclusiveLock on the table, its TOAST relation (if there
+ * is one), all its indexes, so that we can swap the files.
+ */
+ LockRelationOid(old_table_oid, AccessExclusiveLock);
+
+ /*
+ * Lock all indexes now, not only the clustering one: all indexes need to
+ * have their files swapped. While doing that, store their relation
+ * references in an array, to handle predicate locks below.
+ */
+ ind_refs_p = ind_refs = palloc_array(Relation, list_length(ind_oids_old));
+ nind = 0;
+ foreach_oid(ind_oid, ind_oids_old)
+ {
+ Relation index;
+
+ index = index_open(ind_oid, AccessExclusiveLock);
+
+ /*
+ * TODO 1) Do we need to check if ALTER INDEX was executed since the
+ * new index was created in build_new_indexes()? 2) Specifically for
+ * the clustering index, should check_index_is_clusterable() be called
+ * here? (Not sure about the latter: ShareUpdateExclusiveLock on the
+ * table probably blocks all commands that affect the result of
+ * check_index_is_clusterable().)
+ */
+ *ind_refs_p = index;
+ ind_refs_p++;
+ nind++;
+ }
+
+ /*
+ * Lock the OldHeap's TOAST relation exclusively - again, the lock is
+ * needed to swap the files.
+ */
+ if (OidIsValid(OldHeap->rd_rel->reltoastrelid))
+ LockRelationOid(OldHeap->rd_rel->reltoastrelid, AccessExclusiveLock);
+
+ /*
+ * Tuples and pages of the old heap will be gone, but the heap will stay.
+ */
+ TransferPredicateLocksToHeapRelation(OldHeap);
+ /* The same for indexes. */
+ for (int i = 0; i < nind; i++)
+ {
+ Relation index = ind_refs[i];
+
+ TransferPredicateLocksToHeapRelation(index);
+
+ /*
+ * References to indexes on the old relation are not needed anymore,
+ * however locks stay till the end of the transaction.
+ */
+ index_close(index, NoLock);
+ }
+ pfree(ind_refs);
+
+ /*
+ * Flush anything we see in WAL, to make sure that all changes committed
+ * while we were waiting for the exclusive lock are available for
+ * decoding. This should not be necessary if all backends had
+ * synchronous_commit set, but we can't rely on this setting.
+ *
+ * Unfortunately, GetInsertRecPtr() may lag behind the actual insert
+ * position, and GetLastImportantRecPtr() points at the start of the last
+ * record rather than at the end. Thus the simplest way to determine the
+ * insert position is to insert a dummy record and use its LSN.
+ *
+ * XXX Consider using GetLastImportantRecPtr() and adding the size of the
+ * last record (plus the total size of all the page headers the record
+ * spans)?
+ */
+ XLogBeginInsert();
+ XLogRegisterData(&dummy_rec_data, 1);
+ wal_insert_ptr = XLogInsert(RM_XLOG_ID, XLOG_NOOP);
+ XLogFlush(wal_insert_ptr);
+ end_of_wal = GetFlushRecPtr(NULL);
+
+ /*
+ * Apply the concurrent changes again. Indicate that the decoding worker
+ * won't be needed anymore.
+ */
+ process_concurrent_changes(end_of_wal, &chgcxt, true);
+
+ /* Remember info about rel before closing OldHeap */
+ relpersistence = OldHeap->rd_rel->relpersistence;
+ is_system_catalog = IsSystemRelation(OldHeap);
+
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+
+ /*
+ * Even ShareUpdateExclusiveLock should have prevented others from
+ * creating / dropping indexes (even using the CONCURRENTLY option), so we
+ * do not need to check whether the lists match.
+ */
+ forboth(lc, ind_oids_old, lc2, ind_oids_new)
+ {
+ Oid ind_old = lfirst_oid(lc);
+ Oid ind_new = lfirst_oid(lc2);
+ Oid mapped_tables[4];
+
+ /* Zero out possible results from swapped_relation_files */
+ memset(mapped_tables, 0, sizeof(mapped_tables));
+
+ swap_relation_files(ind_old, ind_new,
+ (old_table_oid == RelationRelationId),
+ false, /* swap_toast_by_content */
+ true,
+ InvalidTransactionId,
+ InvalidMultiXactId,
+ mapped_tables);
+
+#ifdef USE_ASSERT_CHECKING
+
+ /*
+ * Concurrent processing is not supported for system relations, so
+ * there should be no mapped tables.
+ */
+ for (int i = 0; i < 4; i++)
+ Assert(mapped_tables[i] == 0);
+#endif
+ }
+
+ /* The new indexes must be visible for deletion. */
+ CommandCounterIncrement();
+
+ /* Close the old heap but keep lock until transaction commit. */
+ table_close(OldHeap, NoLock);
+ /* Close the new heap. (We didn't have to open its indexes). */
+ table_close(NewHeap, NoLock);
+
+ /* Cleanup what we don't need anymore. (And close the identity index.) */
+ release_change_context(&chgcxt);
+
+ /*
+ * Swap the relations and their TOAST relations and TOAST indexes. This
+ * also drops the new relation and its indexes.
+ *
+ * (System catalogs are currently not supported.)
+ */
+ Assert(!is_system_catalog);
+ finish_heap_swap(old_table_oid, new_table_oid,
+ is_system_catalog,
+ false, /* swap_toast_by_content */
+ false,
+ true,
+ false, /* reindex */
+ frozenXid, cutoffMulti,
+ relpersistence);
+}
+
+/*
+ * Build indexes on NewHeap according to those on OldHeap.
+ *
+ * OldIndexes is the list of index OIDs on OldHeap. The contained indexes end
+ * up locked using ShareUpdateExclusiveLock.
+ *
+ * A list of OIDs of the corresponding indexes created on NewHeap is
+ * returned. The order of items does match, so we can use these arrays to swap
+ * index storage.
+ */
+static List *
+build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
+{
+ List *result = NIL;
+
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+
+ foreach_oid(ind_oid, OldIndexes)
+ {
+ Oid ind_oid_new;
+ char *newName;
+ Relation ind;
+
+ ind = index_open(ind_oid, ShareUpdateExclusiveLock);
+
+ newName = ChooseRelationName(get_rel_name(ind_oid),
+ NULL,
+ "repacknew",
+ get_rel_namespace(ind->rd_index->indrelid),
+ false);
+ ind_oid_new = index_create_copy(NewHeap, false, ind_oid,
+ ind->rd_rel->reltablespace, newName);
+ result = lappend_oid(result, ind_oid_new);
+
+ index_close(ind, NoLock);
+ }
+
+ return result;
+}
+
+/*
+ * Try to start a background worker to perform logical decoding of data
+ * changes applied to relation while REPACK CONCURRENTLY is copying its
+ * contents to a new table.
+ */
+static void
+start_repack_decoding_worker(Oid relid)
+{
+ Size size;
+ dsm_segment *seg;
+ DecodingWorkerShared *shared;
+ shm_mq *mq;
+ shm_mq_handle *mqh;
+ BackgroundWorker bgw;
+
+ /* Setup shared memory. */
+ size = BUFFERALIGN(offsetof(DecodingWorkerShared, error_queue)) +
+ BUFFERALIGN(REPACK_ERROR_QUEUE_SIZE);
+ seg = dsm_create(size, 0);
+ shared = (DecodingWorkerShared *) dsm_segment_address(seg);
+ shared->lsn_upto = InvalidXLogRecPtr;
+ shared->done = false;
+ SharedFileSetInit(&shared->sfs, seg);
+ shared->last_exported = -1;
+ SpinLockInit(&shared->mutex);
+ shared->dbid = MyDatabaseId;
+
+ /*
+ * This is the UserId set in cluster_rel(). Security context shouldn't be
+ * needed for decoding worker.
+ */
+ shared->roleid = GetUserId();
+ shared->relid = relid;
+ ConditionVariableInit(&shared->cv);
+ shared->backend_proc = MyProc;
+ shared->backend_pid = MyProcPid;
+ shared->backend_proc_number = MyProcNumber;
+
+ mq = shm_mq_create((char *) BUFFERALIGN(shared->error_queue),
+ REPACK_ERROR_QUEUE_SIZE);
+ shm_mq_set_receiver(mq, MyProc);
+ mqh = shm_mq_attach(mq, seg, NULL);
+
+ memset(&bgw, 0, sizeof(bgw));
+ snprintf(bgw.bgw_name, BGW_MAXLEN,
+ "REPACK decoding worker for relation \"%s\"",
+ get_rel_name(relid));
+ snprintf(bgw.bgw_type, BGW_MAXLEN, "REPACK decoding worker");
+ bgw.bgw_flags = BGWORKER_SHMEM_ACCESS |
+ BGWORKER_BACKEND_DATABASE_CONNECTION;
+ bgw.bgw_start_time = BgWorkerStart_RecoveryFinished;
+ bgw.bgw_restart_time = BGW_NEVER_RESTART;
+ snprintf(bgw.bgw_library_name, MAXPGPATH, "postgres");
+ snprintf(bgw.bgw_function_name, BGW_MAXLEN, "RepackWorkerMain");
+ bgw.bgw_main_arg = UInt32GetDatum(dsm_segment_handle(seg));
+ bgw.bgw_notify_pid = MyProcPid;
+
+ decoding_worker = palloc0_object(DecodingWorker);
+ if (!RegisterDynamicBackgroundWorker(&bgw, &decoding_worker->handle))
+ ereport(ERROR,
+ errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
+ errmsg("out of background worker slots"),
+ errhint("You might need to increase \"%s\".", "max_worker_processes"));
+
+ decoding_worker->seg = seg;
+ decoding_worker->error_mqh = mqh;
+
+ /*
+ * The decoding setup must be done before the caller can have XID assigned
+ * for any reason, otherwise the worker might end up in a deadlock,
+ * waiting for the caller's transaction to end. Therefore wait here until
+ * the worker indicates that it has the logical decoding initialized.
+ */
+ ConditionVariablePrepareToSleep(&shared->cv);
+ for (;;)
+ {
+ bool initialized;
+
+ SpinLockAcquire(&shared->mutex);
+ initialized = shared->initialized;
+ SpinLockRelease(&shared->mutex);
+
+ if (initialized)
+ break;
+
+ ConditionVariableSleep(&shared->cv, WAIT_EVENT_REPACK_WORKER_EXPORT);
+ }
+ ConditionVariableCancelSleep();
+}
+
+/*
+ * Stop the decoding worker and cleanup the related resources.
+ *
+ * The worker stops on its own when it knows there is no more work to do, but
+ * we need to stop it explicitly at least on ERROR in the launching backend.
+ */
+static void
+stop_repack_decoding_worker(void)
+{
+ BgwHandleStatus status;
+
+ /* Haven't reached the worker startup? */
+ if (decoding_worker == NULL)
+ return;
+
+ /* Could not register the worker? */
+ if (decoding_worker->handle == NULL)
+ return;
+
+ TerminateBackgroundWorker(decoding_worker->handle);
+ /* The worker should really exit before the REPACK command does. */
+ HOLD_INTERRUPTS();
+ status = WaitForBackgroundWorkerShutdown(decoding_worker->handle);
+ RESUME_INTERRUPTS();
+
+ if (status == BGWH_POSTMASTER_DIED)
+ ereport(FATAL,
+ errcode(ERRCODE_ADMIN_SHUTDOWN),
+ errmsg("postmaster exited during REPACK command"));
+
+ shm_mq_detach(decoding_worker->error_mqh);
+
+ /*
+ * If we could not cancel the current sleep due to ERROR, do that before
+ * we detach from the shared memory the condition variable is located in.
+ * If we did not, the bgworker ERROR handling code would try and fail
+ * badly.
+ */
+ ConditionVariableCancelSleep();
+
+ dsm_detach(decoding_worker->seg);
+ pfree(decoding_worker);
+ decoding_worker = NULL;
+}
+
+/*
+ * Get the initial snapshot from the decoding worker.
+ */
+static Snapshot
+get_initial_snapshot(DecodingWorker *worker)
+{
+ DecodingWorkerShared *shared;
+ char fname[MAXPGPATH];
+ BufFile *file;
+ Size snap_size;
+ char *snap_space;
+ Snapshot snapshot;
+
+ shared = (DecodingWorkerShared *) dsm_segment_address(worker->seg);
+
+ /*
+ * The worker needs to initialize the logical decoding, which usually
+ * takes some time. Therefore it makes sense to prepare for the sleep
+ * first.
+ */
+ ConditionVariablePrepareToSleep(&shared->cv);
+ for (;;)
+ {
+ int last_exported;
+
+ SpinLockAcquire(&shared->mutex);
+ last_exported = shared->last_exported;
+ SpinLockRelease(&shared->mutex);
+
+ /*
+ * Has the worker exported the file we are waiting for?
+ */
+ if (last_exported == WORKER_FILE_SNAPSHOT)
+ break;
+
+ ConditionVariableSleep(&shared->cv, WAIT_EVENT_REPACK_WORKER_EXPORT);
+ }
+ ConditionVariableCancelSleep();
+
+ /* Read the snapshot from a file. */
+ DecodingWorkerFileName(fname, shared->relid, WORKER_FILE_SNAPSHOT);
+ file = BufFileOpenFileSet(&shared->sfs.fs, fname, O_RDONLY, false);
+ BufFileReadExact(file, &snap_size, sizeof(snap_size));
+ snap_space = (char *) palloc(snap_size);
+ BufFileReadExact(file, snap_space, snap_size);
+ BufFileClose(file);
+
+ /* Restore it. */
+ snapshot = RestoreSnapshot(snap_space);
+ pfree(snap_space);
+
+ return snapshot;
+}
+
+/*
+ * Generate worker's file name into 'fname', which must be of size MAXPGPATH.
+ * If relations of the same 'relid' happen to be processed at the same time,
+ * they must be from different databases and therefore different backends must
+ * be involved.
+ */
+void
+DecodingWorkerFileName(char *fname, Oid relid, uint32 seq)
+{
+ /* The PID is already present in the fileset name, so we needn't add it */
+ snprintf(fname, MAXPGPATH, "%u-%u", relid, seq);
+}
+
+/*
+ * Handle receipt of an interrupt indicating a repack worker message.
+ *
+ * Note: this is called within a signal handler! All we can do is set
+ * a flag that will cause the next CHECK_FOR_INTERRUPTS() to invoke
+ * ProcessRepackMessages().
+ */
+void
+HandleRepackMessageInterrupt(void)
+{
+ InterruptPending = true;
+ RepackMessagePending = true;
+ SetLatch(MyLatch);
+}
+
+/*
+ * Process any queued protocol messages received from the repack worker.
+ */
+void
+ProcessRepackMessages(void)
+{
+ MemoryContext oldcontext;
+ static MemoryContext hpm_context = NULL;
+
+ /*
+ * Nothing to do if we haven't launched the worker yet or have already
+ * terminated it.
+ */
+ if (decoding_worker == NULL)
+ return;
+
+ /*
+ * This is invoked from ProcessInterrupts(), and since some of the
+ * functions it calls contain CHECK_FOR_INTERRUPTS(), there is a potential
+ * for recursive calls if more signals are received while this runs. It's
+ * unclear that recursive entry would be safe, and it doesn't seem useful
+ * even if it is safe, so let's block interrupts until done.
+ */
+ HOLD_INTERRUPTS();
+
+ /*
+ * Moreover, CurrentMemoryContext might be pointing almost anywhere. We
+ * don't want to risk leaking data into long-lived contexts, so let's do
+ * our work here in a private context that we can reset on each use.
+ */
+ if (hpm_context == NULL) /* first time through? */
+ hpm_context = AllocSetContextCreate(TopMemoryContext,
+ "ProcessRepackMessages",
+ ALLOCSET_DEFAULT_SIZES);
+ else
+ MemoryContextReset(hpm_context);
+
+ oldcontext = MemoryContextSwitchTo(hpm_context);
+
+ /* OK to process messages. Reset the flag saying there are more to do. */
+ RepackMessagePending = false;
+
+ /*
+ * Read as many messages as we can from the worker, but stop when no more
+ * messages can be read from the worker without blocking.
+ */
+ while (true)
+ {
+ shm_mq_result res;
+ Size nbytes;
+ void *data;
+
+ res = shm_mq_receive(decoding_worker->error_mqh, &nbytes,
+ &data, true);
+ if (res == SHM_MQ_WOULD_BLOCK)
+ break;
+ else if (res == SHM_MQ_SUCCESS)
+ {
+ StringInfoData msg;
+
+ initStringInfo(&msg);
+ appendBinaryStringInfo(&msg, data, nbytes);
+ ProcessRepackMessage(&msg);
+ pfree(msg.data);
+ }
+ else
+ {
+ /*
+ * The decoding worker is special in that it exits as soon as it
+ * has its work done. Thus the DETACHED result code is fine.
+ */
+ Assert(res == SHM_MQ_DETACHED);
+
+ break;
+ }
+ }
+
+ MemoryContextSwitchTo(oldcontext);
+
+ /* Might as well clear the context on our way out */
+ MemoryContextReset(hpm_context);
+
+ RESUME_INTERRUPTS();
+}
+
+/*
+ * Process a single protocol message received from a single parallel worker.
+ */
+static void
+ProcessRepackMessage(StringInfo msg)
+{
+ char msgtype;
+
+ msgtype = pq_getmsgbyte(msg);
+
+ switch (msgtype)
+ {
+ case PqMsg_ErrorResponse:
+ case PqMsg_NoticeResponse:
+ {
+ ErrorData edata;
+
+ /* Parse ErrorResponse or NoticeResponse. */
+ pq_parse_errornotice(msg, &edata);
+
+ /* Death of a worker isn't enough justification for suicide. */
+ edata.elevel = Min(edata.elevel, ERROR);
+
+ /*
+ * Add a context line to show that this is a message propagated
+ * from the worker. Otherwise, it can sometimes be confusing
+ * to understand what actually happened.
+ */
+ if (edata.context)
+ edata.context = psprintf("%s\n%s", edata.context,
+ _("REPACK decoding worker"));
+ else
+ edata.context = pstrdup(_("REPACK decoding worker"));
+
+ /* Rethrow error or print notice. */
+ ThrowErrorData(&edata);
+
+ break;
+ }
+
+ default:
+ {
+ elog(ERROR, "unrecognized message type received from decoding worker: %c (message length %d bytes)",
+ msgtype, msg->len);
+ }
+ }
+}
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 81a55a33ef2..539969d6eef 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -893,6 +893,7 @@ static void
refresh_by_heap_swap(Oid matviewOid, Oid OIDNewHeap, char relpersistence)
{
finish_heap_swap(matviewOid, OIDNewHeap, false, false, true, true,
+ true, /* reindex */
RecentXmin, ReadNextMultiXactId(), relpersistence);
}
diff --git a/src/backend/commands/meson.build b/src/backend/commands/meson.build
index 90c7e37a429..4a0e6f0002b 100644
--- a/src/backend/commands/meson.build
+++ b/src/backend/commands/meson.build
@@ -39,6 +39,7 @@ backend_sources += files(
'proclang.c',
'propgraphcmds.c',
'publicationcmds.c',
+ 'repack_worker.c',
'schemacmds.c',
'seclabel.c',
'sequence.c',
diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c
new file mode 100644
index 00000000000..35ab1a25c42
--- /dev/null
+++ b/src/backend/commands/repack_worker.c
@@ -0,0 +1,546 @@
+/*-------------------------------------------------------------------------
+ *
+ * repack_worker.c
+ * Implementation of the background worker for ad-hoc logical decoding
+ * during REPACK (CONCURRENTLY).
+ *
+ *
+ * Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/repack_worker.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/table.h"
+#include "access/xlog_internal.h"
+#include "access/xlogutils.h"
+#include "access/xlogwait.h"
+#include "commands/cluster.h"
+#include "commands/repack_internal.h"
+#include "libpq/pqmq.h"
+#include "replication/snapbuild.h"
+#include "storage/ipc.h"
+#include "storage/proc.h"
+#include "tcop/tcopprot.h"
+#include "utils/memutils.h"
+
+#define REPL_PLUGIN_NAME "pgoutput_repack"
+
+static void repack_worker_internal(dsm_segment *seg);
+static void RepackWorkerShutdown(int code, Datum arg);
+static LogicalDecodingContext *repack_setup_logical_decoding(Oid relid);
+static void repack_cleanup_logical_decoding(LogicalDecodingContext *ctx);
+static void export_initial_snapshot(Snapshot snapshot,
+ DecodingWorkerShared *shared);
+static bool decode_concurrent_changes(LogicalDecodingContext *ctx,
+ DecodingWorkerShared *shared);
+
+
+/* Is this process a REPACK worker? */
+static bool is_repack_worker = false;
+
+/* The WAL segment being decoded. */
+static XLogSegNo repack_current_segment = 0;
+
+/*
+ * Keep track of the table we're processing, to skip logical decoding of data
+ * from other relations.
+ */
+static RelFileLocator repacked_rel_locator = {.relNumber = InvalidOid};
+static RelFileLocator repacked_rel_toast_locator = {.relNumber = InvalidOid};
+
+
+/* REPACK decoding worker entry point */
+void
+RepackWorkerMain(Datum main_arg)
+{
+ dsm_segment *seg;
+ DecodingWorkerShared *shared;
+ shm_mq *mq;
+ shm_mq_handle *mqh;
+
+ is_repack_worker = true;
+
+ /*
+ * Override the default bgworker_die() with die() so we can use
+ * CHECK_FOR_INTERRUPTS().
+ */
+ pqsignal(SIGTERM, die);
+ BackgroundWorkerUnblockSignals();
+
+ seg = dsm_attach(DatumGetUInt32(main_arg));
+ if (seg == NULL)
+ ereport(ERROR,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("could not map dynamic shared memory segment"));
+
+ shared = (DecodingWorkerShared *) dsm_segment_address(seg);
+
+ /* Arrange to signal the leader if we exit. */
+ backend_pid = shared->backend_pid;
+ backend_proc_number = shared->backend_proc_number;
+ before_shmem_exit(RepackWorkerShutdown, PointerGetDatum(seg));
+
+ /*
+ * Join locking group - see the comments around the call of
+ * start_repack_decoding_worker().
+ */
+ if (!BecomeLockGroupMember(shared->backend_proc, backend_pid))
+ /* The leader is not running anymore. */
+ return;
+
+ /*
+ * Setup a queue to send error messages to the backend that launched this
+ * worker.
+ */
+ mq = (shm_mq *) (char *) BUFFERALIGN(shared->error_queue);
+ shm_mq_set_sender(mq, MyProc);
+ mqh = shm_mq_attach(mq, seg, NULL);
+ pq_redirect_to_shm_mq(seg, mqh);
+ pq_set_parallel_leader(shared->backend_pid,
+ shared->backend_proc_number);
+
+ /* Connect to the database. */
+ BackgroundWorkerInitializeConnectionByOid(shared->dbid, shared->roleid, 0);
+
+ repack_worker_internal(seg);
+}
+
+static void
+repack_worker_internal(dsm_segment *seg)
+{
+ DecodingWorkerShared *shared;
+ LogicalDecodingContext *decoding_ctx;
+ SharedFileSet *sfs;
+ Snapshot snapshot;
+
+ /*
+ * Transaction is needed to open relation, and it also provides us with a
+ * resource owner.
+ */
+ StartTransactionCommand();
+
+ shared = (DecodingWorkerShared *) dsm_segment_address(seg);
+
+ /*
+ * Not sure the spinlock is needed here - the backend should not change
+ * anything in the shared memory until we have serialized the snapshot.
+ */
+ SpinLockAcquire(&shared->mutex);
+ Assert(!XLogRecPtrIsValid(shared->lsn_upto));
+ sfs = &shared->sfs;
+ SpinLockRelease(&shared->mutex);
+
+ SharedFileSetAttach(sfs, seg);
+
+ /*
+ * Prepare to capture the concurrent data changes ourselves.
+ */
+ decoding_ctx = repack_setup_logical_decoding(shared->relid);
+
+ /* Announce that we're ready. */
+ SpinLockAcquire(&shared->mutex);
+ shared->initialized = true;
+ SpinLockRelease(&shared->mutex);
+ ConditionVariableSignal(&shared->cv);
+
+ /* Build the initial snapshot and export it. */
+ snapshot = SnapBuildInitialSnapshot(decoding_ctx->snapshot_builder, true);
+ export_initial_snapshot(snapshot, shared);
+
+ /*
+ * Only historic snapshots should be used now. Do not let us restrict the
+ * progress of xmin horizon.
+ */
+ InvalidateCatalogSnapshot();
+
+ for (;;)
+ {
+ bool stop = decode_concurrent_changes(decoding_ctx, shared);
+
+ if (stop)
+ break;
+
+ }
+
+ /* Cleanup. */
+ repack_cleanup_logical_decoding(decoding_ctx);
+ CommitTransactionCommand();
+}
+
+/*
+ * See ParallelWorkerShutdown for details.
+ */
+static void
+RepackWorkerShutdown(int code, Datum arg)
+{
+ SendProcSignal(backend_pid,
+ PROCSIG_REPACK_MESSAGE,
+ backend_proc_number);
+
+ dsm_detach((dsm_segment *) DatumGetPointer(arg));
+}
+
+bool
+IsRepackWorker(void)
+{
+ return is_repack_worker;
+}
+
+/*
+ * This function is much like pg_create_logical_replication_slot() except that
+ * the new slot is neither released (if anyone else could read changes from
+ * our slot, we could miss changes other backends do while we copy the
+ * existing data into temporary table), nor persisted (it's easier to handle
+ * crash by restarting all the work from scratch).
+ */
+static LogicalDecodingContext *
+repack_setup_logical_decoding(Oid relid)
+{
+ Relation rel;
+ Oid toastrelid;
+ LogicalDecodingContext *ctx;
+ NameData slotname;
+ RepackDecodingState *dstate;
+ MemoryContext oldcxt;
+
+ /*
+ * REPACK CONCURRENTLY is not allowed in a transaction block, so this
+ * should never fire.
+ */
+ Assert(!TransactionIdIsValid(GetTopTransactionIdIfAny()));
+
+ /*
+ * Make sure we can use logical decoding.
+ */
+ CheckSlotPermissions();
+ CheckLogicalDecodingRequirements();
+
+ /*
+ * A single backend should not execute multiple REPACK commands at a time,
+ * so use PID to make the slot unique.
+ *
+ * RS_TEMPORARY so that the slot gets cleaned up on ERROR.
+ */
+ snprintf(NameStr(slotname), NAMEDATALEN, "repack_%d", MyProcPid);
+ ReplicationSlotCreate(NameStr(slotname), true, RS_TEMPORARY, false, false,
+ false);
+
+ EnsureLogicalDecodingEnabled();
+
+ /*
+ * Neither prepare_write nor do_write callback nor update_progress is
+ * useful for us.
+ */
+ ctx = CreateInitDecodingContext(REPL_PLUGIN_NAME,
+ NIL,
+ true,
+ InvalidXLogRecPtr,
+ XL_ROUTINE(.page_read = read_local_xlog_page,
+ .segment_open = wal_segment_open,
+ .segment_close = wal_segment_close),
+ NULL, NULL, NULL);
+
+ /*
+ * We don't have control on setting fast_forward, so at least check it.
+ */
+ Assert(!ctx->fast_forward);
+
+ /* Avoid logical decoding of other relations. */
+ rel = table_open(relid, AccessShareLock);
+ repacked_rel_locator = rel->rd_locator;
+ toastrelid = rel->rd_rel->reltoastrelid;
+ if (OidIsValid(toastrelid))
+ {
+ Relation toastrel;
+
+ /* Avoid logical decoding of other TOAST relations. */
+ toastrel = table_open(toastrelid, AccessShareLock);
+ repacked_rel_toast_locator = toastrel->rd_locator;
+ table_close(toastrel, AccessShareLock);
+ }
+ table_close(rel, AccessShareLock);
+
+ DecodingContextFindStartpoint(ctx);
+
+ /*
+ * decode_concurrent_changes() needs non-blocking callback.
+ */
+ ctx->reader->routine.page_read = read_local_xlog_page_no_wait;
+
+ /* Some WAL records should have been read. */
+ Assert(ctx->reader->EndRecPtr != InvalidXLogRecPtr);
+
+ /*
+ * Initialize repack_current_segment so that we can notice WAL segment
+ * boundaries.
+ */
+ XLByteToSeg(ctx->reader->EndRecPtr, repack_current_segment,
+ wal_segment_size);
+
+ /* Our private state belongs to the decoding context. */
+ oldcxt = MemoryContextSwitchTo(ctx->context);
+
+ /*
+ * read_local_xlog_page_no_wait() needs to be able to indicate the end of
+ * WAL.
+ */
+ ctx->reader->private_data = palloc0_object(ReadLocalXLogPageNoWaitPrivate);
+ dstate = palloc0_object(RepackDecodingState);
+ MemoryContextSwitchTo(oldcxt);
+
+#ifdef USE_ASSERT_CHECKING
+ dstate->relid = relid;
+#endif
+
+ dstate->change_cxt = AllocSetContextCreate(ctx->context,
+ "REPACK - change",
+ ALLOCSET_DEFAULT_SIZES);
+
+ /* The file will be set as soon as we have it opened. */
+ dstate->file = NULL;
+
+ /*
+ * Memory context and resource owner for long-lived resources.
+ */
+ dstate->worker_cxt = CurrentMemoryContext;
+ dstate->worker_resowner = CurrentResourceOwner;
+
+ ctx->output_writer_private = dstate;
+
+ return ctx;
+}
+
+static void
+repack_cleanup_logical_decoding(LogicalDecodingContext *ctx)
+{
+ RepackDecodingState *dstate;
+
+ dstate = (RepackDecodingState *) ctx->output_writer_private;
+ if (dstate->slot)
+ ExecDropSingleTupleTableSlot(dstate->slot);
+
+ FreeDecodingContext(ctx);
+ ReplicationSlotDropAcquired();
+}
+
+/*
+ * Make snapshot available to the backend that launched the decoding worker.
+ */
+static void
+export_initial_snapshot(Snapshot snapshot, DecodingWorkerShared *shared)
+{
+ char fname[MAXPGPATH];
+ BufFile *file;
+ Size snap_size;
+ char *snap_space;
+
+ snap_size = EstimateSnapshotSpace(snapshot);
+ snap_space = (char *) palloc(snap_size);
+ SerializeSnapshot(snapshot, snap_space);
+ FreeSnapshot(snapshot);
+
+ DecodingWorkerFileName(fname, shared->relid, shared->last_exported + 1);
+ file = BufFileCreateFileSet(&shared->sfs.fs, fname);
+ /* To make restoration easier, write the snapshot size first. */
+ BufFileWrite(file, &snap_size, sizeof(snap_size));
+ BufFileWrite(file, snap_space, snap_size);
+ pfree(snap_space);
+ BufFileClose(file);
+
+ /* Increase the counter to tell the backend that the file is available. */
+ SpinLockAcquire(&shared->mutex);
+ shared->last_exported++;
+ SpinLockRelease(&shared->mutex);
+ ConditionVariableSignal(&shared->cv);
+}
+
+/*
+ * Decode logical changes from the WAL sequence and store them to a file.
+ *
+ * If true is returned, there is no more work for the worker.
+ */
+static bool
+decode_concurrent_changes(LogicalDecodingContext *ctx,
+ DecodingWorkerShared *shared)
+{
+ RepackDecodingState *dstate;
+ XLogRecPtr lsn_upto;
+ bool done;
+ char fname[MAXPGPATH];
+
+ dstate = (RepackDecodingState *) ctx->output_writer_private;
+
+ /* Open the output file. */
+ DecodingWorkerFileName(fname, shared->relid, shared->last_exported + 1);
+ dstate->file = BufFileCreateFileSet(&shared->sfs.fs, fname);
+
+ SpinLockAcquire(&shared->mutex);
+ lsn_upto = shared->lsn_upto;
+ done = shared->done;
+ SpinLockRelease(&shared->mutex);
+
+ while (true)
+ {
+ XLogRecord *record;
+ XLogSegNo segno_new;
+ char *errm = NULL;
+ XLogRecPtr end_lsn;
+
+ CHECK_FOR_INTERRUPTS();
+
+ record = XLogReadRecord(ctx->reader, &errm);
+ if (record)
+ {
+ LogicalDecodingProcessRecord(ctx, ctx->reader);
+
+ /*
+ * If WAL segment boundary has been crossed, inform the decoding
+ * system that the catalog_xmin can advance.
+ *
+ * TODO Does it make sense to confirm more often? Segment size
+ * seems appropriate for restart_lsn (because less than a segment
+ * cannot be recycled anyway), however more frequent checks might
+ * be beneficial for catalog_xmin.
+ */
+ end_lsn = ctx->reader->EndRecPtr;
+ XLByteToSeg(end_lsn, segno_new, wal_segment_size);
+ if (segno_new != repack_current_segment)
+ {
+ LogicalConfirmReceivedLocation(end_lsn);
+ elog(DEBUG1, "REPACK: confirmed receive location %X/%X",
+ (uint32) (end_lsn >> 32), (uint32) end_lsn);
+ repack_current_segment = segno_new;
+ }
+ }
+ else
+ {
+ ReadLocalXLogPageNoWaitPrivate *priv;
+
+ if (errm)
+ ereport(ERROR,
+ errmsg("%s", errm));
+
+ /*
+ * In the decoding loop we do not want to get blocked when there
+ * is no more WAL available, otherwise the loop would become
+ * uninterruptible.
+ */
+ priv = (ReadLocalXLogPageNoWaitPrivate *) ctx->reader->private_data;
+ if (priv->end_of_wal)
+ /* Do not miss the end of WAL condition next time. */
+ priv->end_of_wal = false;
+ else
+ ereport(ERROR,
+ errmsg("could not read WAL record"));
+ }
+
+ /*
+ * Whether we could read new record or not, keep checking if
+ * 'lsn_upto' was specified.
+ */
+ if (!XLogRecPtrIsValid(lsn_upto))
+ {
+ SpinLockAcquire(&shared->mutex);
+ lsn_upto = shared->lsn_upto;
+ /* 'done' should be set at the same time as 'lsn_upto' */
+ done = shared->done;
+ SpinLockRelease(&shared->mutex);
+ }
+ if (XLogRecPtrIsValid(lsn_upto) &&
+ ctx->reader->EndRecPtr >= lsn_upto)
+ break;
+
+ if (record == NULL)
+ {
+ int64 timeout = 0;
+ WaitLSNResult res;
+
+ /*
+ * Before we retry reading, wait until new WAL is flushed.
+ *
+ * There is a race condition such that the backend executing
+ * REPACK determines 'lsn_upto', but before it sets the shared
+ * variable, we reach the end of WAL. In that case we'd need to
+ * wait until the next WAL flush (unrelated to REPACK). Although
+ * that should not be a problem in a busy system, it might be
+ * noticeable in other cases, including regression tests (which
+ * are not necessarily executed in parallel). Therefore it makes
+ * sense to use timeout.
+ *
+ * If lsn_upto is valid, WAL records having LSN lower than that
+ * should already have been flushed to disk.
+ */
+ if (!XLogRecPtrIsValid(lsn_upto))
+ timeout = 100L;
+ res = WaitForLSN(WAIT_LSN_TYPE_PRIMARY_FLUSH,
+ ctx->reader->EndRecPtr + 1,
+ timeout);
+ if (res != WAIT_LSN_RESULT_SUCCESS &&
+ res != WAIT_LSN_RESULT_TIMEOUT)
+ ereport(ERROR,
+ errmsg("waiting for WAL failed"));
+ }
+ }
+
+ /*
+ * Close the file so we can make it available to the backend.
+ */
+ BufFileClose(dstate->file);
+ dstate->file = NULL;
+ SpinLockAcquire(&shared->mutex);
+ shared->lsn_upto = InvalidXLogRecPtr;
+ shared->last_exported++;
+ SpinLockRelease(&shared->mutex);
+ ConditionVariableSignal(&shared->cv);
+
+ return done;
+}
+
+/*
+ * Does the WAL record contain a data change that this backend does not need
+ * to decode on behalf of REPACK (CONCURRENTLY)?
+ */
+bool
+change_useless_for_repack(XLogRecordBuffer *buf)
+{
+ XLogReaderState *r = buf->record;
+ RelFileLocator locator;
+
+ /* TOAST locator should not be set unless the main is. */
+ Assert(!OidIsValid(repacked_rel_toast_locator.relNumber) ||
+ OidIsValid(repacked_rel_locator.relNumber));
+
+ /*
+ * Backends not involved in REPACK (CONCURRENTLY) should not do the
+ * filtering.
+ */
+ if (!OidIsValid(repacked_rel_locator.relNumber))
+ return false;
+
+ /*
+ * If the record does not contain the block 0, it's probably not INSERT /
+ * UPDATE / DELETE. In any case, we do not have enough information to
+ * filter the change out.
+ */
+ if (!XLogRecGetBlockTagExtended(r, 0, &locator, NULL, NULL, NULL))
+ return false;
+
+ /*
+ * Decode the change if it belongs to the table we are repacking, or if it
+ * belongs to its TOAST relation.
+ */
+ if (RelFileLocatorEquals(locator, repacked_rel_locator))
+ return false;
+ if (OidIsValid(repacked_rel_toast_locator.relNumber) &&
+ RelFileLocatorEquals(locator, repacked_rel_toast_locator))
+ return false;
+
+ /* Filter out changes of other tables. */
+ return true;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8b4ebc6f226..1f61bc8520d 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -6058,6 +6058,7 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode,
finish_heap_swap(tab->relid, OIDNewHeap,
false, false, true,
!OidIsValid(tab->newTableSpace),
+ true, /* reindex */
RecentXmin,
ReadNextMultiXactId(),
persistence);
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 766a518c7a1..4f00c697bfb 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -127,7 +127,7 @@ static void vac_truncate_clog(TransactionId frozenXID,
TransactionId lastSaneFrozenXid,
MultiXactId lastSaneMinMulti);
static bool vacuum_rel(Oid relid, RangeVar *relation, VacuumParams params,
- BufferAccessStrategy bstrategy);
+ BufferAccessStrategy bstrategy, bool isTopLevel);
static double compute_parallel_delay(void);
static VacOptValue get_vacoptval_from_boolean(DefElem *def);
static bool vac_tid_reaped(ItemPointer itemptr, void *state);
@@ -630,7 +630,8 @@ vacuum(List *relations, const VacuumParams params, BufferAccessStrategy bstrateg
if (params.options & VACOPT_VACUUM)
{
- if (!vacuum_rel(vrel->oid, vrel->relation, params, bstrategy))
+ if (!vacuum_rel(vrel->oid, vrel->relation, params, bstrategy,
+ isTopLevel))
continue;
}
@@ -2004,7 +2005,7 @@ vac_truncate_clog(TransactionId frozenXID,
*/
static bool
vacuum_rel(Oid relid, RangeVar *relation, VacuumParams params,
- BufferAccessStrategy bstrategy)
+ BufferAccessStrategy bstrategy, bool isTopLevel)
{
LOCKMODE lmode;
Relation rel;
@@ -2295,7 +2296,7 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams params,
/* VACUUM FULL is a variant of REPACK; see cluster.c */
cluster_rel(REPACK_COMMAND_VACUUMFULL, rel, InvalidOid,
- &cluster_params);
+ &cluster_params, isTopLevel);
/* cluster_rel closes the relation, but keeps lock */
rel = NULL;
@@ -2338,7 +2339,8 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams params,
toast_vacuum_params.options |= VACOPT_PROCESS_MAIN;
toast_vacuum_params.toast_parent = relid;
- vacuum_rel(toast_relid, NULL, toast_vacuum_params, bstrategy);
+ vacuum_rel(toast_relid, NULL, toast_vacuum_params, bstrategy,
+ isTopLevel);
}
/*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 76728f08734..61b12c052d2 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2338,7 +2338,7 @@ lreplace:
0,
estate->es_snapshot,
estate->es_crosscheck_snapshot,
- true /* wait for commit */ ,
+ true,
&context->tmfd, &updateCxt->lockmode,
&updateCxt->updateIndexes);
diff --git a/src/backend/libpq/pqmq.c b/src/backend/libpq/pqmq.c
index 22e5164adbf..1000b7bb06e 100644
--- a/src/backend/libpq/pqmq.c
+++ b/src/backend/libpq/pqmq.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "access/parallel.h"
+#include "commands/cluster.h"
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
#include "libpq/pqmq.h"
@@ -177,6 +178,10 @@ mq_putmessage(char msgtype, const char *s, size_t len)
SendProcSignal(pq_mq_parallel_leader_pid,
PROCSIG_PARALLEL_APPLY_MESSAGE,
pq_mq_parallel_leader_proc_number);
+ else if (IsRepackWorker())
+ SendProcSignal(pq_mq_parallel_leader_pid,
+ PROCSIG_REPACK_MESSAGE,
+ pq_mq_parallel_leader_proc_number);
else
{
Assert(IsParallelWorker());
diff --git a/src/backend/meson.build b/src/backend/meson.build
index 4f5292d8f88..5e3cfe2d6f8 100644
--- a/src/backend/meson.build
+++ b/src/backend/meson.build
@@ -219,5 +219,6 @@ pg_test_mod_args = pg_mod_args + {
subdir('jit/llvm')
subdir('replication/libpqwalreceiver')
subdir('replication/pgoutput')
+subdir('replication/pgoutput_repack')
subdir('snowball')
subdir('utils/mb/conversion_procs')
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index f2a62489d9c..423ce9d8709 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -13,6 +13,7 @@
#include "postgres.h"
#include "access/parallel.h"
+#include "commands/cluster.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
@@ -136,6 +137,10 @@ static const struct
.fn_name = "ParallelWorkerMain",
.fn_addr = ParallelWorkerMain
},
+ {
+ .fn_name = "RepackWorkerMain",
+ .fn_addr = RepackWorkerMain
+ },
{
.fn_name = "SequenceSyncWorkerMain",
.fn_addr = SequenceSyncWorkerMain
@@ -144,7 +149,6 @@ static const struct
.fn_name = "TableSyncWorkerMain",
.fn_addr = TableSyncWorkerMain
},
-
};
/* Private functions. */
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index 3c027bcb2f7..f172c0b2b40 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -33,6 +33,7 @@
#include "access/xlogreader.h"
#include "access/xlogrecord.h"
#include "catalog/pg_control.h"
+#include "commands/cluster.h"
#include "replication/decode.h"
#include "replication/logical.h"
#include "replication/message.h"
@@ -420,7 +421,8 @@ heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
{
case XLOG_HEAP2_MULTI_INSERT:
if (SnapBuildProcessChange(builder, xid, buf->origptr) &&
- !ctx->fast_forward)
+ !ctx->fast_forward &&
+ !change_useless_for_repack(buf))
DecodeMultiInsert(ctx, buf);
break;
case XLOG_HEAP2_NEW_CID:
@@ -465,6 +467,15 @@ heap_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
TransactionId xid = XLogRecGetXid(buf->record);
SnapBuild *builder = ctx->snapshot_builder;
+ /*
+ * XXX Should we return here if change_useless_for_repack() returns true,
+ * instead of calling the function below? Unlike the fast-forward case, we
+ * shouldn't need the base snapshot for the containing transaction until
+ * we receive a change that belongs to the table being REPACKed. Thus it
+ * should be fine to skip SnapBuildProcessChange(), and therefore
+ * reorderbuffer.c can create the transaction later.
+ */
+
ReorderBufferProcessXid(ctx->reorder, xid, buf->origptr);
/*
@@ -482,7 +493,8 @@ heap_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
{
case XLOG_HEAP_INSERT:
if (SnapBuildProcessChange(builder, xid, buf->origptr) &&
- !ctx->fast_forward)
+ !ctx->fast_forward &&
+ !change_useless_for_repack(buf))
DecodeInsert(ctx, buf);
break;
@@ -494,19 +506,22 @@ heap_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
case XLOG_HEAP_HOT_UPDATE:
case XLOG_HEAP_UPDATE:
if (SnapBuildProcessChange(builder, xid, buf->origptr) &&
- !ctx->fast_forward)
+ !ctx->fast_forward &&
+ !change_useless_for_repack(buf))
DecodeUpdate(ctx, buf);
break;
case XLOG_HEAP_DELETE:
if (SnapBuildProcessChange(builder, xid, buf->origptr) &&
- !ctx->fast_forward)
+ !ctx->fast_forward &&
+ !change_useless_for_repack(buf))
DecodeDelete(ctx, buf);
break;
case XLOG_HEAP_TRUNCATE:
if (SnapBuildProcessChange(builder, xid, buf->origptr) &&
- !ctx->fast_forward)
+ !ctx->fast_forward &&
+ !change_useless_for_repack(buf))
DecodeTruncate(ctx, buf);
break;
@@ -522,7 +537,8 @@ heap_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
case XLOG_HEAP_CONFIRM:
if (SnapBuildProcessChange(builder, xid, buf->origptr) &&
- !ctx->fast_forward)
+ !ctx->fast_forward &&
+ !change_useless_for_repack(buf))
DecodeSpecConfirm(ctx, buf);
break;
@@ -1019,6 +1035,15 @@ DecodeDelete(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
xlrec = (xl_heap_delete *) XLogRecGetData(r);
+ /*
+ * Ignore changes which are considered useless for logical decoding.
+ * Currently such changes are created by REPACK (CONCURRENTLY) when
+ * replays DELETE commands on the new table (which is not yet visible to
+ * other transactions).
+ */
+ if (xlrec->flags & XLH_DELETE_NO_LOGICAL)
+ return;
+
/* only interested in our database */
XLogRecGetBlockTag(r, 0, &target_locator, NULL, NULL);
if (target_locator.dbOid != ctx->slot->data.database)
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 603a2b94d05..7651b187418 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -194,7 +194,11 @@ StartupDecodingContext(List *output_plugin_options,
ctx->slot = slot;
- ctx->reader = XLogReaderAllocate(wal_segment_size, NULL, xl_routine, ctx);
+ /*
+ * TODO A separate patch for PG core, unless there's really a reason to
+ * pass ctx for private_data (May extensions expect ctx?).
+ */
+ ctx->reader = XLogReaderAllocate(wal_segment_size, NULL, xl_routine, NULL);
if (!ctx->reader)
ereport(ERROR,
(errcode(ERRCODE_OUT_OF_MEMORY),
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index 37f0c6028bd..9cf499ce7c6 100644
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -440,7 +440,7 @@ SnapBuildBuildSnapshot(SnapBuild *builder)
* for loading in different transaction.
*/
Snapshot
-SnapBuildInitialSnapshot(SnapBuild *builder)
+SnapBuildInitialSnapshot(SnapBuild *builder, bool repack)
{
Snapshot snap;
TransactionId xid;
@@ -448,7 +448,7 @@ SnapBuildInitialSnapshot(SnapBuild *builder)
TransactionId *newxip;
int newxcnt = 0;
- Assert(XactIsoLevel == XACT_REPEATABLE_READ);
+ Assert(XactIsoLevel == XACT_REPEATABLE_READ || repack);
Assert(builder->building_full_snapshot);
/* don't allow older snapshots */
@@ -526,6 +526,11 @@ SnapBuildInitialSnapshot(SnapBuild *builder)
snap->xcnt = newxcnt;
snap->xip = newxip;
+ /*
+ * FreeSnapshot() is more appropriate for REPACK than counting references.
+ */
+ snap->copied = repack;
+
return snap;
}
@@ -558,7 +563,7 @@ SnapBuildExportSnapshot(SnapBuild *builder)
XactIsoLevel = XACT_REPEATABLE_READ;
XactReadOnly = true;
- snap = SnapBuildInitialSnapshot(builder);
+ snap = SnapBuildInitialSnapshot(builder, false);
/*
* now that we've built a plain snapshot, make it active and use the
diff --git a/src/backend/replication/pgoutput_repack/Makefile b/src/backend/replication/pgoutput_repack/Makefile
new file mode 100644
index 00000000000..4efeb713b70
--- /dev/null
+++ b/src/backend/replication/pgoutput_repack/Makefile
@@ -0,0 +1,32 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for src/backend/replication/pgoutput_repack
+#
+# IDENTIFICATION
+# src/backend/replication/pgoutput_repack
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/replication/pgoutput_repack
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = \
+ $(WIN32RES) \
+ pgoutput_repack.o
+PGFILEDESC = "pgoutput_repack - logical replication output plugin for REPACK command"
+NAME = pgoutput_repack
+
+all: all-shared-lib
+
+include $(top_srcdir)/src/Makefile.shlib
+
+install: all installdirs install-lib
+
+installdirs: installdirs-lib
+
+uninstall: uninstall-lib
+
+clean distclean: clean-lib
+ rm -f $(OBJS)
diff --git a/src/backend/replication/pgoutput_repack/meson.build b/src/backend/replication/pgoutput_repack/meson.build
new file mode 100644
index 00000000000..6a88c0fb08d
--- /dev/null
+++ b/src/backend/replication/pgoutput_repack/meson.build
@@ -0,0 +1,18 @@
+# Copyright (c) 2022-2026, PostgreSQL Global Development Group
+
+pgoutput_repack_sources = files(
+ 'pgoutput_repack.c',
+)
+
+if host_system == 'windows'
+ pgoutput_repack_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'pgoutput_repack',
+ '--FILEDESC', 'pgoutput_repack - logical replication output plugin for REPACK command',])
+endif
+
+pgoutput_repack = shared_module('pgoutput_repack',
+ pgoutput_repack_sources,
+ kwargs: pg_mod_args,
+)
+
+backend_targets += pgoutput_repack
diff --git a/src/backend/replication/pgoutput_repack/pgoutput_repack.c b/src/backend/replication/pgoutput_repack/pgoutput_repack.c
new file mode 100644
index 00000000000..221260ca0a8
--- /dev/null
+++ b/src/backend/replication/pgoutput_repack/pgoutput_repack.c
@@ -0,0 +1,290 @@
+/*-------------------------------------------------------------------------
+ *
+ * pgoutput_repack.c
+ * Logical Replication output plugin for REPACK command
+ *
+ * Copyright (c) 2012-2026, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/pgoutput_repack/pgoutput_repack.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/detoast.h"
+#include "commands/repack_internal.h"
+#include "replication/snapbuild.h"
+#include "utils/memutils.h"
+
+PG_MODULE_MAGIC;
+
+static void repack_startup(LogicalDecodingContext *ctx,
+ OutputPluginOptions *opt, bool is_init);
+static void repack_shutdown(LogicalDecodingContext *ctx);
+static void repack_begin_txn(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn);
+static void repack_commit_txn(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr commit_lsn);
+static void repack_process_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ Relation rel, ReorderBufferChange *change);
+static void repack_store_change(LogicalDecodingContext *ctx, Relation relation,
+ ConcurrentChangeKind kind, HeapTuple tuple);
+
+void
+_PG_output_plugin_init(OutputPluginCallbacks *cb)
+{
+ cb->startup_cb = repack_startup;
+ cb->begin_cb = repack_begin_txn;
+ cb->change_cb = repack_process_change;
+ cb->commit_cb = repack_commit_txn;
+ cb->shutdown_cb = repack_shutdown;
+}
+
+
+/* initialize this plugin */
+static void
+repack_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
+ bool is_init)
+{
+ ctx->output_plugin_private = NULL;
+
+ /* Probably unnecessary, as we don't use the SQL interface ... */
+ opt->output_type = OUTPUT_PLUGIN_BINARY_OUTPUT;
+
+ if (ctx->output_plugin_options != NIL)
+ {
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("this plugin does not expect any options"));
+ }
+}
+
+static void
+repack_shutdown(LogicalDecodingContext *ctx)
+{
+}
+
+/*
+ * As we don't release the slot during processing of particular table, there's
+ * no room for SQL interface, even for debugging purposes. Therefore we need
+ * neither OutputPluginPrepareWrite() nor OutputPluginWrite() in the plugin
+ * callbacks. (Although we might want to write custom callbacks, this API
+ * seems to be unnecessarily generic for our purposes.)
+ */
+
+/* BEGIN callback */
+static void
+repack_begin_txn(LogicalDecodingContext *ctx, ReorderBufferTXN *txn)
+{
+}
+
+/* COMMIT callback */
+static void
+repack_commit_txn(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr commit_lsn)
+{
+}
+
+/*
+ * Callback for individual changed tuples
+ */
+static void
+repack_process_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ Relation relation, ReorderBufferChange *change)
+{
+ RepackDecodingState *private PG_USED_FOR_ASSERTS_ONLY =
+ (RepackDecodingState *) ctx->output_writer_private;
+
+ /* Changes of other relation should not have been decoded. */
+ Assert(RelationGetRelid(relation) == private->relid);
+
+ /* Decode entry depending on its type */
+ switch (change->action)
+ {
+ case REORDER_BUFFER_CHANGE_INSERT:
+ {
+ HeapTuple newtuple;
+
+ newtuple = change->data.tp.newtuple;
+
+ /*
+ * Identity checks in the main function should have made this
+ * impossible.
+ */
+ if (newtuple == NULL)
+ elog(ERROR, "incomplete insert info");
+
+ repack_store_change(ctx, relation, CHANGE_INSERT, newtuple);
+ }
+ break;
+ case REORDER_BUFFER_CHANGE_UPDATE:
+ {
+ HeapTuple oldtuple,
+ newtuple;
+
+ oldtuple = change->data.tp.oldtuple;
+ newtuple = change->data.tp.newtuple;
+
+ if (newtuple == NULL)
+ elog(ERROR, "incomplete update info");
+
+ if (oldtuple != NULL)
+ repack_store_change(ctx, relation, CHANGE_UPDATE_OLD, oldtuple);
+
+ repack_store_change(ctx, relation, CHANGE_UPDATE_NEW, newtuple);
+ }
+ break;
+ case REORDER_BUFFER_CHANGE_DELETE:
+ {
+ HeapTuple oldtuple;
+
+ oldtuple = change->data.tp.oldtuple;
+
+ if (oldtuple == NULL)
+ elog(ERROR, "incomplete delete info");
+
+ repack_store_change(ctx, relation, CHANGE_DELETE, oldtuple);
+ }
+ break;
+ default:
+
+ /*
+ * Should not come here. This includes TRUNCATE of the table being
+ * processed. heap_decode() cannot check the file locator easily,
+ * but we assume that TRUNCATE uses AccessExclusiveLock on the
+ * table so it should not occur during REPACK (CONCURRENTLY).
+ */
+ Assert(false);
+ break;
+ }
+}
+
+/*
+ * Write the given tuple, with the given change kind, to the repack spill
+ * file. Later, the repack decoding worker can read these and replay
+ * the operations on the new copy of the table.
+ *
+ * For each change affecting the table being repacked, we store enough
+ * information about each tuple in it, so that it can be replayed in the
+ * new copy of the table.
+ *
+ * XXX for DELETE and the UPDATE OLD tuples, we could store just the
+ * replication identity instead of the full tuple.
+ */
+static void
+repack_store_change(LogicalDecodingContext *ctx, Relation relation,
+ ConcurrentChangeKind kind, HeapTuple tuple)
+{
+ RepackDecodingState *dstate;
+ MemoryContext oldcxt;
+ BufFile *file;
+ List *attrs_ext = NIL;
+ int natt_ext;
+
+ dstate = (RepackDecodingState *) ctx->output_writer_private;
+ file = dstate->file;
+
+ /* Store the change kind. */
+ BufFileWrite(file, &kind, 1);
+
+ /* Use a frequently-reset context to avoid dealing with leaks manually */
+ oldcxt = MemoryContextSwitchTo(dstate->change_cxt);
+
+ /*
+ * If the tuple contains "external indirect" attributes, we need to write
+ * the contents to the file because we have no control over that memory.
+ */
+ if (HeapTupleHasExternal(tuple))
+ {
+ TupleDesc desc = RelationGetDescr(relation);
+ TupleTableSlot *slot;
+
+ /* Initialize the slot, if not done already */
+ if (dstate->slot == NULL)
+ {
+ ResourceOwner saveResourceOwner;
+
+ MemoryContextSwitchTo(dstate->worker_cxt);
+ saveResourceOwner = CurrentResourceOwner;
+ CurrentResourceOwner = dstate->worker_resowner;
+ dstate->slot = MakeSingleTupleTableSlot(desc, &TTSOpsHeapTuple);
+ MemoryContextSwitchTo(dstate->change_cxt);
+ CurrentResourceOwner = saveResourceOwner;
+ }
+
+ slot = dstate->slot;
+ ExecStoreHeapTuple(tuple, slot, false);
+
+ /*
+ * Loop over all attributes, and find out which ones we need to spill
+ * separately, to wit: each one that's a non-null varlena and stored
+ * out of line.
+ */
+ for (int i = 0; i < desc->natts; i++)
+ {
+ CompactAttribute *attr = TupleDescCompactAttr(desc, i);
+ varlena *varlen;
+
+ if (attr->attisdropped || attr->attlen != -1 ||
+ slot_attisnull(slot, i + 1))
+ continue;
+
+ slot_getsomeattrs(slot, i + 1);
+
+ /*
+ * This is a non-null varlena datum, but we only care if it's
+ * out-of-line
+ */
+ varlen = (varlena *) DatumGetPointer(slot->tts_values[i]);
+ if (!VARATT_IS_EXTERNAL(varlen))
+ continue;
+
+ /*
+ * We spill any indirect-external attributes separately from the
+ * heap tuple. Anything else is written as is.
+ */
+ if (VARATT_IS_EXTERNAL_INDIRECT(varlen))
+ attrs_ext = lappend(attrs_ext, varlen);
+ else
+ {
+ /*
+ * Logical decoding should not produce "external expanded"
+ * attributes (those actually should never appear on disk), so
+ * only TOASTed attribute can be seen here.
+ *
+ * We get here if the table has external values but only
+ * in-line values are being updated now.
+ */
+ Assert(VARATT_IS_EXTERNAL_ONDISK(varlen));
+ }
+ }
+
+ ExecClearTuple(slot);
+ }
+
+ /*
+ * First, write the original heap tuple, prefixed by its length. Note
+ * that the external-toast tag for each toasted attribute will be present
+ * in what we write, so that we know where to restore each one later.
+ */
+ BufFileWrite(file, &tuple->t_len, sizeof(tuple->t_len));
+ BufFileWrite(file, tuple->t_data, tuple->t_len);
+
+ /* Then, write the number of external attributes we found. */
+ natt_ext = list_length(attrs_ext);
+ BufFileWrite(file, &natt_ext, sizeof(natt_ext));
+
+ /* Finally, the attributes themselves, if any */
+ foreach_ptr(varlena, attr_val, attrs_ext)
+ {
+ attr_val = detoast_external_attr(attr_val);
+ BufFileWrite(file, attr_val, VARSIZE_ANY(attr_val));
+ /* These attributes could be large, so free them right away */
+ pfree(attr_val);
+ }
+
+ /* Cleanup. */
+ MemoryContextSwitchTo(oldcxt);
+ MemoryContextReset(dstate->change_cxt);
+}
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 66507e9c2dd..fbbe09135bf 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -1342,7 +1342,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd)
{
Snapshot snap;
- snap = SnapBuildInitialSnapshot(ctx->snapshot_builder);
+ snap = SnapBuildInitialSnapshot(ctx->snapshot_builder, false);
RestoreTransactionSnapshot(snap, MyProc);
}
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 7e017c8d53b..dd980145ced 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -19,6 +19,7 @@
#include "access/parallel.h"
#include "commands/async.h"
+#include "commands/cluster.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
@@ -700,6 +701,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
+ if (CheckProcSignal(PROCSIG_REPACK_MESSAGE))
+ HandleRepackMessageInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT))
HandleRecoveryConflictInterrupt();
diff --git a/src/backend/storage/lmgr/generate-lwlocknames.pl b/src/backend/storage/lmgr/generate-lwlocknames.pl
index b49007167b0..2e7f1054e62 100644
--- a/src/backend/storage/lmgr/generate-lwlocknames.pl
+++ b/src/backend/storage/lmgr/generate-lwlocknames.pl
@@ -162,7 +162,7 @@ while (<$lwlocklist>)
die
"$wait_event_lwlocks[$lwlock_count] defined in wait_event_names.txt but "
- . " missing from lwlocklist.h"
+ . "missing from lwlocklist.h"
if $lwlock_count < scalar @wait_event_lwlocks;
die
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 10be60011ad..b2936d503b6 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -36,6 +36,7 @@
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/async.h"
+#include "commands/cluster.h"
#include "commands/event_trigger.h"
#include "commands/explain_state.h"
#include "commands/prepare.h"
@@ -3576,6 +3577,9 @@ ProcessInterrupts(void)
if (ParallelApplyMessagePending)
ProcessParallelApplyMessages();
+
+ if (RepackMessagePending)
+ ProcessRepackMessages();
}
/*
diff --git a/src/backend/utils/activity/wait_event_names.txt b/src/backend/utils/activity/wait_event_names.txt
index 6be80d2daad..e2f21349997 100644
--- a/src/backend/utils/activity/wait_event_names.txt
+++ b/src/backend/utils/activity/wait_event_names.txt
@@ -154,6 +154,7 @@ RECOVERY_CONFLICT_SNAPSHOT "Waiting for recovery conflict resolution for a vacuu
RECOVERY_CONFLICT_TABLESPACE "Waiting for recovery conflict resolution for dropping a tablespace."
RECOVERY_END_COMMAND "Waiting for <xref linkend="guc-recovery-end-command"/> to complete."
RECOVERY_PAUSE "Waiting for recovery to be resumed."
+REPACK_WORKER_EXPORT "Waiting for decoding worker to export a new output file."
REPLICATION_ORIGIN_DROP "Waiting for a replication origin to become inactive so it can be dropped."
REPLICATION_SLOT_DROP "Waiting for a replication slot to become inactive so it can be dropped."
RESTORE_COMMAND "Waiting for <xref linkend="guc-restore-command"/> to complete."
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 2e6197f5f35..e0129df3e40 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -215,7 +215,6 @@ static List *exportedSnapshots = NIL;
/* Prototypes for local functions */
static Snapshot CopySnapshot(Snapshot snapshot);
static void UnregisterSnapshotNoOwner(Snapshot snapshot);
-static void FreeSnapshot(Snapshot snapshot);
static void SnapshotResetXmin(void);
/* ResourceOwner callbacks to track snapshot references */
@@ -660,7 +659,7 @@ CopySnapshot(Snapshot snapshot)
* FreeSnapshot
* Free the memory associated with a snapshot.
*/
-static void
+void
FreeSnapshot(Snapshot snapshot)
{
Assert(snapshot->regd_count == 0);
diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c
index 53bf1e21721..1a22f162ce6 100644
--- a/src/bin/psql/tab-complete.in.c
+++ b/src/bin/psql/tab-complete.in.c
@@ -5231,8 +5231,8 @@ match_previous_words(int pattern_id,
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("ANALYZE", "VERBOSE");
- else if (TailMatches("ANALYZE", "VERBOSE"))
+ COMPLETE_WITH("ANALYZE", "CONCURRENTLY", "VERBOSE");
+ else if (TailMatches("ANALYZE", "CONCURRENTLY", "VERBOSE"))
COMPLETE_WITH("ON", "OFF");
}
diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h
index 516806fcca2..fdca7d821c8 100644
--- a/src/include/access/heapam_xlog.h
+++ b/src/include/access/heapam_xlog.h
@@ -104,6 +104,8 @@
#define XLH_DELETE_CONTAINS_OLD_KEY (1<<2)
#define XLH_DELETE_IS_SUPER (1<<3)
#define XLH_DELETE_IS_PARTITION_MOVE (1<<4)
+/* See heap_delete() */
+#define XLH_DELETE_NO_LOGICAL (1<<5)
/* convenience macro for checking whether any form of old tuple was logged */
#define XLH_DELETE_CONTAINS_OLD \
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 6cdd949f58d..9afd0284cc0 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -289,9 +289,10 @@ typedef struct TM_IndexDeleteOp
/* "options" flag bits for table_tuple_delete */
#define TABLE_DELETE_CHANGING_PARTITION (1 << 0)
+#define TABLE_DELETE_NO_LOGICAL (1 << 1)
/* "options" flag bits for table_tuple_update */
-/* XXX none at present */
+#define TABLE_UPDATE_NO_LOGICAL (1 << 0)
/* Typedef for callback function for table_index_build_scan */
typedef void (*IndexBuildCallback) (Relation index,
@@ -660,6 +661,7 @@ typedef struct TableAmRoutine
Relation OldIndex,
bool use_sort,
TransactionId OldestXmin,
+ Snapshot snapshot,
TransactionId *xid_cutoff,
MultiXactId *multi_cutoff,
double *num_tuples,
@@ -1522,6 +1524,7 @@ table_multi_insert(Relation rel, TupleTableSlot **slots, int nslots,
* cid - delete command ID (used for visibility test, and stored into
* cmax if successful)
* crosscheck - if not InvalidSnapshot, also check tuple against this
+ * XXX document options
* wait - true if should wait for any conflicting update to commit/abort
* changingPart - true iff the tuple is being moved to another partition
* table due to an update of the partition key. Otherwise, false.
@@ -1559,7 +1562,12 @@ table_tuple_delete(Relation rel, ItemPointer tid, CommandId cid,
* cid - update command ID (used for visibility test, and stored into
* cmax/cmin if successful)
* crosscheck - if not InvalidSnapshot, also check old tuple against this
- * wait - true if should wait for any conflicting update to commit/abort
+ * options - These allow the caller to specify options that may change the
+ * behavior of the AM. The AM will ignore options that it does not support.
+ * TABLE_UPDATE_WAIT -- set if should wait for any conflicting update to
+ * commit/abort
+ * TABLE_UPDATE_NO_LOGICAL -- force-disables the emitting of logical
+ * decoding information for the tuple.
*
* Output parameters:
* slot - newly constructed tuple data to store
@@ -1721,6 +1729,8 @@ table_relation_copy_data(Relation rel, const RelFileLocator *newrlocator)
* not needed for the relation's AM
* - *xid_cutoff - ditto
* - *multi_cutoff - ditto
+ * - snapshot - if != NULL, ignore data changes done by transactions that this
+ * (MVCC) snapshot considers still in-progress or in the future.
*
* Output parameters:
* - *xid_cutoff - rel's new relfrozenxid value, may be invalid
@@ -1733,6 +1743,7 @@ table_relation_copy_for_cluster(Relation OldTable, Relation NewTable,
Relation OldIndex,
bool use_sort,
TransactionId OldestXmin,
+ Snapshot snapshot,
TransactionId *xid_cutoff,
MultiXactId *multi_cutoff,
double *num_tuples,
@@ -1741,6 +1752,7 @@ table_relation_copy_for_cluster(Relation OldTable, Relation NewTable,
{
OldTable->rd_tableam->relation_copy_for_cluster(OldTable, NewTable, OldIndex,
use_sort, OldestXmin,
+ snapshot,
xid_cutoff, multi_cutoff,
num_tuples, tups_vacuumed,
tups_recently_dead);
diff --git a/src/include/commands/cluster.h b/src/include/commands/cluster.h
index d6b62c747e8..81673ffad1a 100644
--- a/src/include/commands/cluster.h
+++ b/src/include/commands/cluster.h
@@ -13,6 +13,8 @@
#ifndef CLUSTER_H
#define CLUSTER_H
+#include <signal.h>
+
#include "nodes/parsenodes.h"
#include "parser/parse_node.h"
#include "storage/lockdefs.h"
@@ -25,6 +27,7 @@
#define CLUOPT_RECHECK_ISCLUSTERED 0x04 /* recheck relation state for
* indisclustered */
#define CLUOPT_ANALYZE 0x08 /* do an ANALYZE */
+#define CLUOPT_CONCURRENT 0x10 /* allow concurrent data changes */
/* options for CLUSTER */
typedef struct ClusterParams
@@ -32,11 +35,13 @@ typedef struct ClusterParams
uint32 options; /* bitmask of CLUOPT_* */
} ClusterParams;
+extern PGDLLIMPORT volatile sig_atomic_t RepackMessagePending;
+
extern void ExecRepack(ParseState *pstate, RepackStmt *stmt, bool isTopLevel);
extern void cluster_rel(RepackCommand command, Relation OldHeap, Oid indexOid,
- ClusterParams *params);
+ ClusterParams *params, bool isTopLevel);
extern void check_index_is_clusterable(Relation OldHeap, Oid indexOid,
LOCKMODE lockmode);
extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal);
@@ -48,8 +53,17 @@ extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
bool swap_toast_by_content,
bool check_constraints,
bool is_internal,
+ bool reindex,
TransactionId frozenXid,
MultiXactId cutoffMulti,
char newrelpersistence);
+extern void HandleRepackMessageInterrupt(void);
+extern void ProcessRepackMessages(void);
+
+/* in repack_worker.c */
+extern void RepackWorkerMain(Datum main_arg);
+extern bool IsRepackWorker(void);
+
+
#endif /* CLUSTER_H */
diff --git a/src/include/commands/progress.h b/src/include/commands/progress.h
index 9c40772706c..b5b01c1bb6d 100644
--- a/src/include/commands/progress.h
+++ b/src/include/commands/progress.h
@@ -86,10 +86,12 @@
#define PROGRESS_REPACK_PHASE 1
#define PROGRESS_REPACK_INDEX_RELID 2
#define PROGRESS_REPACK_HEAP_TUPLES_SCANNED 3
-#define PROGRESS_REPACK_HEAP_TUPLES_WRITTEN 4
-#define PROGRESS_REPACK_TOTAL_HEAP_BLKS 5
-#define PROGRESS_REPACK_HEAP_BLKS_SCANNED 6
-#define PROGRESS_REPACK_INDEX_REBUILD_COUNT 7
+#define PROGRESS_REPACK_HEAP_TUPLES_INSERTED 4
+#define PROGRESS_REPACK_HEAP_TUPLES_UPDATED 5
+#define PROGRESS_REPACK_HEAP_TUPLES_DELETED 6
+#define PROGRESS_REPACK_TOTAL_HEAP_BLKS 7
+#define PROGRESS_REPACK_HEAP_BLKS_SCANNED 8
+#define PROGRESS_REPACK_INDEX_REBUILD_COUNT 9
/*
* Phases of repack (as advertised via PROGRESS_REPACK_PHASE).
@@ -98,9 +100,10 @@
#define PROGRESS_REPACK_PHASE_INDEX_SCAN_HEAP 2
#define PROGRESS_REPACK_PHASE_SORT_TUPLES 3
#define PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP 4
-#define PROGRESS_REPACK_PHASE_SWAP_REL_FILES 5
-#define PROGRESS_REPACK_PHASE_REBUILD_INDEX 6
-#define PROGRESS_REPACK_PHASE_FINAL_CLEANUP 7
+#define PROGRESS_REPACK_PHASE_CATCH_UP 5
+#define PROGRESS_REPACK_PHASE_SWAP_REL_FILES 6
+#define PROGRESS_REPACK_PHASE_REBUILD_INDEX 7
+#define PROGRESS_REPACK_PHASE_FINAL_CLEANUP 8
/* Progress parameters for CREATE INDEX */
/* 3, 4 and 5 reserved for "waitfor" metrics */
diff --git a/src/include/commands/repack_internal.h b/src/include/commands/repack_internal.h
new file mode 100644
index 00000000000..d3402e0dedd
--- /dev/null
+++ b/src/include/commands/repack_internal.h
@@ -0,0 +1,128 @@
+/*-------------------------------------------------------------------------
+ *
+ * repack_internal.h
+ * header for REPACK internals
+ *
+ * Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994-5, Regents of the University of California
+ *
+ * src/include/commands/repack_internal.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef REPACK_INTERNAL_H
+#define REPACK_INTERNAL_H
+
+#include "nodes/execnodes.h"
+#include "replication/decode.h"
+#include "postmaster/bgworker.h"
+#include "replication/logical.h"
+#include "storage/buffile.h"
+#include "storage/sharedfileset.h"
+#include "storage/shm_mq.h"
+#include "utils/resowner.h"
+
+/*
+ * Stored as a single byte in the output file.
+ */
+typedef char ConcurrentChangeKind;
+
+#define CHANGE_INSERT 'i'
+#define CHANGE_UPDATE_OLD 'u'
+#define CHANGE_UPDATE_NEW 'U'
+#define CHANGE_DELETE 'd'
+
+extern pid_t backend_pid;
+extern ProcNumber backend_proc_number;
+
+/*
+ * Logical decoding state.
+ *
+ * The output plugin uses it to store the data changes that it decodes from
+ * WAL while the table contents is being copied to a new storage.
+ */
+typedef struct RepackDecodingState
+{
+#ifdef USE_ASSERT_CHECKING
+ /* The relation whose changes we're decoding. */
+ Oid relid;
+#endif
+
+ /* Per-change memory context. */
+ MemoryContext change_cxt;
+
+ /* A tuple slot used to pass tuples back and forth */
+ TupleTableSlot *slot;
+
+ /*
+ * Memory context and resource owner of the decoding worker's transaction.
+ */
+ MemoryContext worker_cxt;
+ ResourceOwner worker_resowner;
+
+ /* The current output file. */
+ BufFile *file;
+} RepackDecodingState;
+
+/*
+ * Layout of shared memory used for communication between backend and the
+ * worker that performs logical decoding of data changes
+ */
+typedef struct DecodingWorkerShared
+{
+ /* Is the decoding initialized? */
+ bool initialized;
+
+ /*
+ * Once the worker has reached this LSN, it should close the current
+ * output file and either create a new one or exit, according to the field
+ * 'done'. If the value is InvalidXLogRecPtr, the worker should decode all
+ * the WAL available and keep checking this field. It is ok if the worker
+ * had already decoded records whose LSN is >= lsn_upto before this field
+ * has been set.
+ */
+ XLogRecPtr lsn_upto;
+
+ /* Exit after closing the current file? */
+ bool done;
+
+ /* The output is stored here. */
+ SharedFileSet sfs;
+
+ /* Number of the last file exported by the worker. */
+ int last_exported;
+
+ /* Synchronize access to the fields above. */
+ slock_t mutex;
+
+ /* Database to connect to. */
+ Oid dbid;
+
+ /* Role to connect as. */
+ Oid roleid;
+
+ /* Decode data changes of this relation. */
+ Oid relid;
+
+ /* The backend uses this to wait for the worker. */
+ ConditionVariable cv;
+
+ /* Info to signal the backend. */
+ PGPROC *backend_proc;
+ pid_t backend_pid;
+ ProcNumber backend_proc_number;
+
+ /*
+ * Memory the queue is located in.
+ *
+ * For considerations on the value see the comments of
+ * PARALLEL_ERROR_QUEUE_SIZE.
+ */
+#define REPACK_ERROR_QUEUE_SIZE 16384
+ char error_queue[FLEXIBLE_ARRAY_MEMBER];
+} DecodingWorkerShared;
+
+extern void DecodingWorkerFileName(char *fname, Oid relid, uint32 seq);
+
+
+#endif /* REPACK_INTERNAL_H */
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index 49f00fc48b8..02b5393474c 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -31,4 +31,8 @@ extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
+/* in commands/cluster.c */
+extern bool change_useless_for_repack(XLogRecordBuffer *buf);
+
+
#endif
diff --git a/src/include/replication/snapbuild.h b/src/include/replication/snapbuild.h
index ccded021433..2b84f0058f0 100644
--- a/src/include/replication/snapbuild.h
+++ b/src/include/replication/snapbuild.h
@@ -72,7 +72,7 @@ extern void FreeSnapshotBuilder(SnapBuild *builder);
extern void SnapBuildSnapDecRefcount(Snapshot snap);
-extern Snapshot SnapBuildInitialSnapshot(SnapBuild *builder);
+extern Snapshot SnapBuildInitialSnapshot(SnapBuild *builder, bool repack);
extern const char *SnapBuildExportSnapshot(SnapBuild *builder);
extern void SnapBuildClearExportedSnapshot(void);
extern void SnapBuildResetExportedSnapshotState(void);
diff --git a/src/include/storage/lockdefs.h b/src/include/storage/lockdefs.h
index b73bb5618e6..3785b009808 100644
--- a/src/include/storage/lockdefs.h
+++ b/src/include/storage/lockdefs.h
@@ -36,8 +36,8 @@ typedef int LOCKMODE;
#define AccessShareLock 1 /* SELECT */
#define RowShareLock 2 /* SELECT FOR UPDATE/FOR SHARE */
#define RowExclusiveLock 3 /* INSERT, UPDATE, DELETE */
-#define ShareUpdateExclusiveLock 4 /* VACUUM (non-FULL), ANALYZE, CREATE
- * INDEX CONCURRENTLY */
+#define ShareUpdateExclusiveLock 4 /* VACUUM (non-exclusive), ANALYZE, CREATE
+ * INDEX CONCURRENTLY, REPACK CONCURRENTLY */
#define ShareLock 5 /* CREATE INDEX (WITHOUT CONCURRENTLY) */
#define ShareRowExclusiveLock 6 /* like EXCLUSIVE MODE, but allows ROW
* SHARE */
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 348fba53a93..a944ee0d211 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -36,6 +36,7 @@ typedef enum
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
+ PROCSIG_REPACK_MESSAGE, /* Message from repack worker */
PROCSIG_RECOVERY_CONFLICT, /* backend is blocking recovery, check
* PGPROC->pendingRecoveryConflicts for the
* reason */
diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h
index 1c550096393..55ecc2c2054 100644
--- a/src/include/utils/snapmgr.h
+++ b/src/include/utils/snapmgr.h
@@ -78,6 +78,8 @@ extern Snapshot GetTransactionSnapshot(void);
extern Snapshot GetLatestSnapshot(void);
extern void SnapshotSetCommandId(CommandId curcid);
+extern void FreeSnapshot(Snapshot snapshot);
+
extern Snapshot GetCatalogSnapshot(Oid relid);
extern Snapshot GetNonHistoricCatalogSnapshot(Oid relid);
extern void InvalidateCatalogSnapshot(void);
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index a41d781f8c9..2cd7d87c533 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -14,6 +14,8 @@ REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
ISOLATION = basic \
inplace \
+ repack \
+ repack_toast \
syscache-update-pruned \
heap_lock_update
diff --git a/src/test/modules/injection_points/expected/repack.out b/src/test/modules/injection_points/expected/repack.out
new file mode 100644
index 00000000000..b575e9052ee
--- /dev/null
+++ b/src/test/modules/injection_points/expected/repack.out
@@ -0,0 +1,113 @@
+Parsed test spec with 2 sessions
+
+starting permutation: wait_before_lock change_existing change_new change_subxact1 change_subxact2 check2 wakeup_before_lock check1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step wait_before_lock:
+ REPACK (CONCURRENTLY) repack_test USING INDEX repack_test_pkey;
+ <waiting ...>
+step change_existing:
+ UPDATE repack_test SET i=10 where i=1;
+ UPDATE repack_test SET j=20 where i=2;
+ UPDATE repack_test SET i=30 where i=3;
+ UPDATE repack_test SET i=40 where i=30;
+ DELETE FROM repack_test WHERE i=4;
+
+step change_new:
+ INSERT INTO repack_test(i, j) VALUES (5, 5), (6, 6), (7, 7), (8, 8);
+ UPDATE repack_test SET i=50 where i=5;
+ UPDATE repack_test SET j=60 where i=6;
+ DELETE FROM repack_test WHERE i=7;
+
+step change_subxact1:
+ BEGIN;
+ INSERT INTO repack_test(i, j) VALUES (100, 100);
+ SAVEPOINT s1;
+ UPDATE repack_test SET i=101 where i=100;
+ SAVEPOINT s2;
+ UPDATE repack_test SET i=102 where i=101;
+ COMMIT;
+
+step change_subxact2:
+ BEGIN;
+ SAVEPOINT s1;
+ INSERT INTO repack_test(i, j) VALUES (110, 110);
+ ROLLBACK TO SAVEPOINT s1;
+ INSERT INTO repack_test(i, j) VALUES (110, 111);
+ COMMIT;
+
+step check2:
+ INSERT INTO relfilenodes(node)
+ SELECT relfilenode FROM pg_class WHERE relname='repack_test';
+
+ SELECT i, j FROM repack_test ORDER BY i, j;
+
+ INSERT INTO data_s2(i, j)
+ SELECT i, j FROM repack_test;
+
+ i| j
+---+---
+ 2| 20
+ 6| 60
+ 8| 8
+ 10| 1
+ 40| 3
+ 50| 5
+102|100
+110|111
+(8 rows)
+
+step wakeup_before_lock:
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step wait_before_lock: <... completed>
+step check1:
+ INSERT INTO relfilenodes(node)
+ SELECT relfilenode FROM pg_class WHERE relname='repack_test';
+
+ SELECT count(DISTINCT node) FROM relfilenodes;
+
+ SELECT i, j FROM repack_test ORDER BY i, j;
+
+ INSERT INTO data_s1(i, j)
+ SELECT i, j FROM repack_test;
+
+ SELECT count(*)
+ FROM data_s1 d1 FULL JOIN data_s2 d2 USING (i, j)
+ WHERE d1.i ISNULL OR d2.i ISNULL;
+
+count
+-----
+ 2
+(1 row)
+
+ i| j
+---+---
+ 2| 20
+ 6| 60
+ 8| 8
+ 10| 1
+ 40| 3
+ 50| 5
+102|100
+110|111
+(8 rows)
+
+count
+-----
+ 0
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
diff --git a/src/test/modules/injection_points/expected/repack_toast.out b/src/test/modules/injection_points/expected/repack_toast.out
new file mode 100644
index 00000000000..b56dde134f8
--- /dev/null
+++ b/src/test/modules/injection_points/expected/repack_toast.out
@@ -0,0 +1,65 @@
+Parsed test spec with 2 sessions
+
+starting permutation: wait_before_lock change check2 wakeup_before_lock check1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step wait_before_lock:
+ REPACK (CONCURRENTLY) repack_test;
+ <waiting ...>
+step change:
+ UPDATE repack_test SET j=get_long_string() where i=2;
+ DELETE FROM repack_test WHERE i=3;
+ INSERT INTO repack_test(i, j) VALUES (4, get_long_string());
+ UPDATE repack_test SET i=3 where i=1;
+
+step check2:
+ INSERT INTO relfilenodes(node)
+ SELECT c2.relfilenode
+ FROM pg_class c1 JOIN pg_class c2 ON c2.oid = c1.oid OR c2.oid = c1.reltoastrelid
+ WHERE c1.relname='repack_test';
+
+ INSERT INTO data_s2(i, j)
+ SELECT i, j FROM repack_test;
+
+step wakeup_before_lock:
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step wait_before_lock: <... completed>
+step check1:
+ INSERT INTO relfilenodes(node)
+ SELECT c2.relfilenode
+ FROM pg_class c1 JOIN pg_class c2 ON c2.oid = c1.oid OR c2.oid = c1.reltoastrelid
+ WHERE c1.relname='repack_test';
+
+ SELECT count(DISTINCT node) FROM relfilenodes;
+
+ INSERT INTO data_s1(i, j)
+ SELECT i, j FROM repack_test;
+
+ SELECT count(*)
+ FROM data_s1 d1 FULL JOIN data_s2 d2 USING (i, j)
+ WHERE d1.i ISNULL OR d2.i ISNULL;
+
+count
+-----
+ 4
+(1 row)
+
+count
+-----
+ 0
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index fcc85414515..a414abb924b 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -45,6 +45,8 @@ tests += {
'specs': [
'basic',
'inplace',
+ 'repack',
+ 'repack_toast',
'syscache-update-pruned',
'heap_lock_update',
],
diff --git a/src/test/modules/injection_points/specs/repack.spec b/src/test/modules/injection_points/specs/repack.spec
new file mode 100644
index 00000000000..d727a9b056b
--- /dev/null
+++ b/src/test/modules/injection_points/specs/repack.spec
@@ -0,0 +1,142 @@
+# REPACK (CONCURRENTLY) ... USING INDEX ...;
+setup
+{
+ CREATE EXTENSION injection_points;
+
+ CREATE TABLE repack_test(i int PRIMARY KEY, j int);
+ INSERT INTO repack_test(i, j) VALUES (1, 1), (2, 2), (3, 3), (4, 4);
+
+ CREATE TABLE relfilenodes(node oid);
+
+ CREATE TABLE data_s1(i int, j int);
+ CREATE TABLE data_s2(i int, j int);
+}
+
+teardown
+{
+ DROP TABLE repack_test;
+ DROP EXTENSION injection_points;
+
+ DROP TABLE relfilenodes;
+ DROP TABLE data_s1;
+ DROP TABLE data_s2;
+}
+
+session s1
+setup
+{
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('repack-concurrently-before-lock', 'wait');
+}
+# Perform the initial load and wait for s2 to do some data changes.
+step wait_before_lock
+{
+ REPACK (CONCURRENTLY) repack_test USING INDEX repack_test_pkey;
+}
+# Check the table from the perspective of s1.
+#
+# Besides the contents, we also check that relfilenode has changed.
+
+# Have each session write the contents into a table and use FULL JOIN to check
+# if the outputs are identical.
+step check1
+{
+ INSERT INTO relfilenodes(node)
+ SELECT relfilenode FROM pg_class WHERE relname='repack_test';
+
+ SELECT count(DISTINCT node) FROM relfilenodes;
+
+ SELECT i, j FROM repack_test ORDER BY i, j;
+
+ INSERT INTO data_s1(i, j)
+ SELECT i, j FROM repack_test;
+
+ SELECT count(*)
+ FROM data_s1 d1 FULL JOIN data_s2 d2 USING (i, j)
+ WHERE d1.i ISNULL OR d2.i ISNULL;
+}
+teardown
+{
+ SELECT injection_points_detach('repack-concurrently-before-lock');
+}
+
+session s2
+# Change the existing data. UPDATE changes both key and non-key columns. Also
+# update one row twice to test whether tuple version generated by this session
+# can be found.
+step change_existing
+{
+ UPDATE repack_test SET i=10 where i=1;
+ UPDATE repack_test SET j=20 where i=2;
+ UPDATE repack_test SET i=30 where i=3;
+ UPDATE repack_test SET i=40 where i=30;
+ DELETE FROM repack_test WHERE i=4;
+}
+# Insert new rows and UPDATE / DELETE some of them. Again, update both key and
+# non-key column.
+step change_new
+{
+ INSERT INTO repack_test(i, j) VALUES (5, 5), (6, 6), (7, 7), (8, 8);
+ UPDATE repack_test SET i=50 where i=5;
+ UPDATE repack_test SET j=60 where i=6;
+ DELETE FROM repack_test WHERE i=7;
+}
+
+# When applying concurrent data changes, we should see the effects of an
+# in-progress subtransaction.
+#
+# XXX Not sure this test is useful now - it was designed for the patch that
+# preserves tuple visibility and which therefore modifies
+# TransactionIdIsCurrentTransactionId().
+step change_subxact1
+{
+ BEGIN;
+ INSERT INTO repack_test(i, j) VALUES (100, 100);
+ SAVEPOINT s1;
+ UPDATE repack_test SET i=101 where i=100;
+ SAVEPOINT s2;
+ UPDATE repack_test SET i=102 where i=101;
+ COMMIT;
+}
+
+# When applying concurrent data changes, we should not see the effects of a
+# rolled back subtransaction.
+#
+# XXX Is this test useful? See above.
+step change_subxact2
+{
+ BEGIN;
+ SAVEPOINT s1;
+ INSERT INTO repack_test(i, j) VALUES (110, 110);
+ ROLLBACK TO SAVEPOINT s1;
+ INSERT INTO repack_test(i, j) VALUES (110, 111);
+ COMMIT;
+}
+
+# Check the table from the perspective of s2.
+step check2
+{
+ INSERT INTO relfilenodes(node)
+ SELECT relfilenode FROM pg_class WHERE relname='repack_test';
+
+ SELECT i, j FROM repack_test ORDER BY i, j;
+
+ INSERT INTO data_s2(i, j)
+ SELECT i, j FROM repack_test;
+}
+step wakeup_before_lock
+{
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+}
+
+# Test if data changes introduced while one session is performing REPACK
+# CONCURRENTLY find their way into the table.
+permutation
+ wait_before_lock
+ change_existing
+ change_new
+ change_subxact1
+ change_subxact2
+ check2
+ wakeup_before_lock
+ check1
diff --git a/src/test/modules/injection_points/specs/repack_toast.spec b/src/test/modules/injection_points/specs/repack_toast.spec
new file mode 100644
index 00000000000..b878b198971
--- /dev/null
+++ b/src/test/modules/injection_points/specs/repack_toast.spec
@@ -0,0 +1,112 @@
+# REPACK (CONCURRENTLY);
+#
+# Test handling of TOAST. At the same time, no tuplesort.
+setup
+{
+ CREATE EXTENSION injection_points;
+
+ -- Return a string that needs to be TOASTed.
+ CREATE FUNCTION get_long_string()
+ RETURNS text
+ LANGUAGE sql as $$
+ SELECT string_agg(chr(65 + trunc(25 * random())::int), '')
+ FROM generate_series(1, 2048) s(x);
+ $$;
+
+ CREATE TABLE repack_test(i int PRIMARY KEY, j text);
+ INSERT INTO repack_test(i, j) VALUES (1, get_long_string()),
+ (2, get_long_string()), (3, get_long_string());
+
+ CREATE TABLE relfilenodes(node oid);
+
+ CREATE TABLE data_s1(i int, j text);
+ CREATE TABLE data_s2(i int, j text);
+}
+
+teardown
+{
+ DROP TABLE repack_test;
+ DROP EXTENSION injection_points;
+ DROP FUNCTION get_long_string();
+
+ DROP TABLE relfilenodes;
+ DROP TABLE data_s1;
+ DROP TABLE data_s2;
+}
+
+session s1
+setup
+{
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('repack-concurrently-before-lock', 'wait');
+}
+# Perform the initial load and wait for s2 to do some data changes.
+step wait_before_lock
+{
+ REPACK (CONCURRENTLY) repack_test;
+}
+# Check the table from the perspective of s1.
+#
+# Besides the contents, we also check that relfilenode has changed.
+
+# Have each session write the contents into a table and use FULL JOIN to check
+# if the outputs are identical.
+step check1
+{
+ INSERT INTO relfilenodes(node)
+ SELECT c2.relfilenode
+ FROM pg_class c1 JOIN pg_class c2 ON c2.oid = c1.oid OR c2.oid = c1.reltoastrelid
+ WHERE c1.relname='repack_test';
+
+ SELECT count(DISTINCT node) FROM relfilenodes;
+
+ INSERT INTO data_s1(i, j)
+ SELECT i, j FROM repack_test;
+
+ SELECT count(*)
+ FROM data_s1 d1 FULL JOIN data_s2 d2 USING (i, j)
+ WHERE d1.i ISNULL OR d2.i ISNULL;
+}
+teardown
+{
+ SELECT injection_points_detach('repack-concurrently-before-lock');
+}
+
+session s2
+step change
+# Separately test UPDATE of both plain ("i") and TOASTed ("j") attribute. In
+# the first case, the new tuple we get from reorderbuffer.c contains "j" as a
+# TOAST pointer, which we need to update so it points to the new heap. In the
+# latter case, we receive "j" as "external indirect" value - here we test that
+# the decoding worker writes the tuple to a file correctly and that the
+# backend executing REPACK manages to restore it.
+{
+ UPDATE repack_test SET j=get_long_string() where i=2;
+ DELETE FROM repack_test WHERE i=3;
+ INSERT INTO repack_test(i, j) VALUES (4, get_long_string());
+ UPDATE repack_test SET i=3 where i=1;
+}
+# Check the table from the perspective of s2.
+step check2
+{
+ INSERT INTO relfilenodes(node)
+ SELECT c2.relfilenode
+ FROM pg_class c1 JOIN pg_class c2 ON c2.oid = c1.oid OR c2.oid = c1.reltoastrelid
+ WHERE c1.relname='repack_test';
+
+ INSERT INTO data_s2(i, j)
+ SELECT i, j FROM repack_test;
+}
+step wakeup_before_lock
+{
+ SELECT injection_points_wakeup('repack-concurrently-before-lock');
+}
+
+# Test if data changes introduced while one session is performing REPACK
+# CONCURRENTLY find their way into the table.
+permutation
+ wait_before_lock
+ change
+ check2
+ wakeup_before_lock
+ check1
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 2b3cf6d8569..84d61f3ab34 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2021,7 +2021,7 @@ pg_stat_progress_cluster| SELECT pid,
phase,
repack_index_relid AS cluster_index_relid,
heap_tuples_scanned,
- heap_tuples_written,
+ (heap_tuples_inserted + heap_tuples_updated) AS heap_tuples_written,
heap_blks_total,
heap_blks_scanned,
index_rebuild_count
@@ -2101,17 +2101,20 @@ pg_stat_progress_repack| SELECT s.pid,
WHEN 2 THEN 'index scanning heap'::text
WHEN 3 THEN 'sorting tuples'::text
WHEN 4 THEN 'writing new heap'::text
- WHEN 5 THEN 'swapping relation files'::text
- WHEN 6 THEN 'rebuilding index'::text
- WHEN 7 THEN 'performing final cleanup'::text
+ WHEN 5 THEN 'catch-up'::text
+ WHEN 6 THEN 'swapping relation files'::text
+ WHEN 7 THEN 'rebuilding index'::text
+ WHEN 8 THEN 'performing final cleanup'::text
ELSE NULL::text
END AS phase,
(s.param3)::oid AS repack_index_relid,
s.param4 AS heap_tuples_scanned,
- s.param5 AS heap_tuples_written,
- s.param6 AS heap_blks_total,
- s.param7 AS heap_blks_scanned,
- s.param8 AS index_rebuild_count
+ s.param5 AS heap_tuples_inserted,
+ s.param6 AS heap_tuples_updated,
+ s.param7 AS heap_tuples_deleted,
+ s.param8 AS heap_blks_total,
+ s.param9 AS heap_blks_scanned,
+ s.param10 AS index_rebuild_count
FROM (pg_stat_get_progress_info('REPACK'::text) s(pid, datid, relid, param1, param2, param3, param4, param5, param6, param7, param8, param9, param10, param11, param12, param13, param14, param15, param16, param17, param18, param19, param20)
LEFT JOIN pg_database d ON ((s.datid = d.oid)));
pg_stat_progress_vacuum| SELECT s.pid,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index c9da1f91cb9..f6e55517b80 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -428,6 +428,7 @@ CatCacheHeader
CatalogId
CatalogIdMapEntry
CatalogIndexState
+ChangeContext
ChangeVarNodes_callback
ChangeVarNodes_context
ChannelName
@@ -505,6 +506,7 @@ CompressFileHandle
CompressionLocation
CompressorState
ComputeXidHorizonsResult
+ConcurrentChangeKind
ConditionVariable
ConditionVariableMinimallyPadded
ConditionalStack
@@ -646,6 +648,8 @@ DeclareCursorStmt
DecodedBkpBlock
DecodedXLogRecord
DecodingOutputState
+DecodingWorker
+DecodingWorkerShared
DefElem
DefElemAction
DefaultACLInfo
@@ -1306,6 +1310,7 @@ IndexElem
IndexFetchHeapData
IndexFetchTableData
IndexInfo
+IndexInsertState
IndexList
IndexOnlyScan
IndexOnlyScanState
@@ -2620,6 +2625,7 @@ ReorderBufferTupleCidKey
ReorderBufferUpdateProgressTxnCB
ReorderTuple
RepackCommand
+RepackDecodingState
RepackStmt
ReparameterizeForeignPathByChild_function
ReplOriginId
--
2.47.3
--2fxmamo6mu2qbxgv
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="v47-0004-repack-code-cleanups.patch"
^ permalink raw reply [nested|flat] 32+ messages in thread
end of thread, other threads:[~2026-03-11 14:16 UTC | newest]
Thread overview: 32+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-11-20 06:26 Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE Guillaume Lelarge <[email protected]>
2024-11-20 15:50 ` Greg Sabino Mullane <[email protected]>
2024-11-20 19:30 ` Guillaume Lelarge <[email protected]>
2024-11-20 21:22 ` David Rowley <[email protected]>
2024-11-20 21:35 ` Bruce Momjian <[email protected]>
2024-11-20 21:54 ` Vik Fearing <[email protected]>
2024-11-21 05:13 ` Laurenz Albe <[email protected]>
2024-12-02 11:41 ` Michael Christofides <[email protected]>
2024-12-03 14:32 ` Robert Haas <[email protected]>
2024-11-25 20:44 ` Robert Haas <[email protected]>
2024-11-25 22:52 ` David Rowley <[email protected]>
2024-11-25 23:12 ` Tom Lane <[email protected]>
2024-11-27 03:02 ` David Rowley <[email protected]>
2024-11-27 16:18 ` Robert Haas <[email protected]>
2024-11-27 09:16 ` Daniel Gustafsson <[email protected]>
2024-12-10 02:57 ` David Rowley <[email protected]>
2024-12-10 12:22 ` Guillaume Lelarge <[email protected]>
2024-12-11 09:38 ` David Rowley <[email protected]>
2024-12-11 11:16 ` David Rowley <[email protected]>
2024-12-11 12:41 ` Guillaume Lelarge <[email protected]>
2024-12-11 14:00 ` jian he <[email protected]>
2024-12-11 20:29 ` Guillaume Lelarge <[email protected]>
2024-12-11 20:55 ` David Rowley <[email protected]>
2024-12-13 15:59 ` Michael Christofides <[email protected]>
2025-02-09 12:05 ` Jelte Fennema-Nio <[email protected]>
2025-05-29 04:28 ` Fujii Masao <[email protected]>
2025-05-30 11:55 ` David Rowley <[email protected]>
2025-05-30 12:43 ` Jelte Fennema-Nio <[email protected]>
2025-05-30 14:36 ` Fujii Masao <[email protected]>
2024-12-11 20:41 ` Jelte Fennema-Nio <[email protected]>
2024-12-11 20:46 ` Guillaume Lelarge <[email protected]>
2026-03-11 14:16 [PATCH v47 3/9] Add CONCURRENTLY option to REPACK command. Antonin Houska <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox