public inbox for [email protected]
help / color / mirror / Atom feedFix Heap Blocks accumulation for Parallel Bitmap Heap Scan
2+ messages / 2 participants
[nested] [flat]
* Fix Heap Blocks accumulation for Parallel Bitmap Heap Scan
@ 2026-04-05 18:51 Lukas Fittl <[email protected]>
2026-04-05 19:08 ` Re: Fix Heap Blocks accumulation for Parallel Bitmap Heap Scan Melanie Plageman <[email protected]>
0 siblings, 1 reply; 2+ messages in thread
From: Lukas Fittl @ 2026-04-05 18:51 UTC (permalink / raw)
To: PostgreSQL Hackers <[email protected]>; +Cc: Melanie Plageman <[email protected]>; Tomas Vondra <[email protected]>; Andres Freund <[email protected]>
Hi,
In another thread [0], Andres noted that Parallel Bitmap Heap Scan and
Parallel Index-Only Scan don't have any test coverage for EXPLAIN
(ANALYZE):
parallel BHS is not covered:
https://coverage.postgresql.org/src/backend/executor/nodeBitmapHeapscan.c.gcov.html#L536
parallel IOS is not covered:
https://coverage.postgresql.org/src/backend/executor/nodeIndexonlyscan.c.gcov.html#L430
In the process of adding that coverage, I found a bug.
This is slightly different than the bug that Tomas and Melanie posted
about on an adjacent thread [1], so starting a new one:
When secondary instrumentation data (that's not WalUsage or
BufferUsage) gets handled by parallel workers, we need to do a lot of
special handling. On top of making sure we copy the information, we
also need to teach explain.c to aggregate the per-worker node
instrumentation, like done for index searches in
show_indexsearches_info:
static void
show_indexsearches_info(PlanState *planstate, ExplainState *es)
{
Plan *plan = planstate->plan;
SharedIndexScanInstrumentation *SharedInfo = NULL;
uint64 nsearches = 0;
if (!es->analyze)
return;
...
/* Next get the sum of the counters set within each and every process */
if (SharedInfo)
{
for (int i = 0; i < SharedInfo->num_workers; ++i)
{
IndexScanInstrumentation *winstrument = &SharedInfo->winstrument[i];
nsearches += winstrument->nsearches;
}
}
ExplainPropertyUInteger("Index Searches", NULL, nsearches, es);
}
Parallel Bitmap Heap Scans were missing such handling in
show_tidbitmap_info, causing the information shown on the plan to only
reflect the Heap Blocks of the leader, not that of the parallel
workers. I think this is inconsistent, and should be fixed.
See attached a patch that fixes that in show_tidbitmap_info and adds
test coverage. See also attached a second patch that adds missing
EXPLAIN ANALYZE test coverage for Parallel Index Only Scans.
Thanks,
Lukas
[0]: https://www.postgresql.org/message-id/57biou6l65r7gr4nunoe6lignz2x6m3w45gihoypaez4pc46di@txj3bakhj66...
[1]: https://www.postgresql.org/message-id/flat/dbd45d67-8fb9-464f-b3ed-6fe185f8c8c9%40vondra.me#8e15b225...
--
Lukas Fittl
Attachments:
[application/x-patch] v1-0001-Parallel-Bitmap-Heap-Scan-Fix-EXPLAIN-reporting-o.patch (5.9K, 2-v1-0001-Parallel-Bitmap-Heap-Scan-Fix-EXPLAIN-reporting-o.patch)
download | inline diff:
From 080407d94ff6d87ecbb1f757c26b34959dc6d19d Mon Sep 17 00:00:00 2001
From: Lukas Fittl <[email protected]>
Date: Sun, 5 Apr 2026 03:39:46 -0700
Subject: [PATCH v1 1/2] Parallel Bitmap Heap Scan: Fix EXPLAIN reporting of
"Heap Blocks"
Fix the missing accumulation of "Heap Blocks" from parallel query workers
to the leader, causing EXPLAIN (ANALYZE) to only show the leader statistics,
significantly undercounting the true value.
Additionally, add a regression test covering EXPLAIN (ANALYZE) of a
Parallel Bitmap Heap Scan, which previously was not tested at all.
Author: Lukas Fittl <[email protected]>
Reviewed-by:
Discussion
---
src/backend/commands/explain.c | 33 +++++++++++++++++++++------
src/test/regress/expected/explain.out | 33 +++++++++++++++++++++++++++
src/test/regress/sql/explain.sql | 31 +++++++++++++++++++++++++
3 files changed, 90 insertions(+), 7 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index e4b70166b0e..b01a2e1e149 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -3924,26 +3924,45 @@ show_indexsearches_info(PlanState *planstate, ExplainState *es)
static void
show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
{
+ uint64 exact_pages;
+ uint64 lossy_pages;
+
if (!es->analyze)
return;
+ /* Start with leader's stats */
+ exact_pages = planstate->stats.exact_pages;
+ lossy_pages = planstate->stats.lossy_pages;
+
+ /* Accumulate worker stats into node-level totals */
+ if (planstate->sinstrument != NULL)
+ {
+ for (int n = 0; n < planstate->sinstrument->num_workers; n++)
+ {
+ BitmapHeapScanInstrumentation *si = &planstate->sinstrument->sinstrument[n];
+
+ exact_pages += si->exact_pages;
+ lossy_pages += si->lossy_pages;
+ }
+ }
+
if (es->format != EXPLAIN_FORMAT_TEXT)
{
ExplainPropertyUInteger("Exact Heap Blocks", NULL,
- planstate->stats.exact_pages, es);
+ exact_pages, es);
ExplainPropertyUInteger("Lossy Heap Blocks", NULL,
- planstate->stats.lossy_pages, es);
+ lossy_pages, es);
}
else
{
- if (planstate->stats.exact_pages > 0 || planstate->stats.lossy_pages > 0)
+ if (exact_pages > 0 || lossy_pages > 0)
{
ExplainIndentText(es);
appendStringInfoString(es->str, "Heap Blocks:");
- if (planstate->stats.exact_pages > 0)
- appendStringInfo(es->str, " exact=" UINT64_FORMAT, planstate->stats.exact_pages);
- if (planstate->stats.lossy_pages > 0)
- appendStringInfo(es->str, " lossy=" UINT64_FORMAT, planstate->stats.lossy_pages);
+ if (exact_pages > 0)
+ appendStringInfo(es->str, " exact=" UINT64_FORMAT, exact_pages);
+ if (lossy_pages > 0)
+ appendStringInfo(es->str, " lossy=" UINT64_FORMAT, lossy_pages);
appendStringInfoChar(es->str, '\n');
}
}
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 7c1f26b182c..58c5a512d74 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -822,3 +822,36 @@ select explain_filter('explain (analyze,buffers off,costs off) select sum(n) ove
(9 rows)
reset work_mem;
+-- Test parallel bitmap heap scan reports per-worker heap block stats.
+CREATE FUNCTION check_parallel_bitmap_heap_scan() RETURNS boolean AS $$
+DECLARE
+ plan_json json;
+ node json;
+BEGIN
+ SET LOCAL enable_seqscan = off;
+ SET LOCAL enable_indexscan = off;
+ SET LOCAL parallel_setup_cost = 0;
+ SET LOCAL parallel_tuple_cost = 0;
+ SET LOCAL min_parallel_table_scan_size = 0;
+ SET LOCAL min_parallel_index_scan_size = 0;
+ SET LOCAL max_parallel_workers_per_gather = 2;
+ SET LOCAL parallel_leader_participation = off;
+
+ EXECUTE 'EXPLAIN (ANALYZE, BUFFERS, COSTS OFF, FORMAT JSON)
+ SELECT count(*) FROM tenk1 WHERE hundred > 1' INTO plan_json;
+
+ node := plan_json->0->'Plan';
+ WHILE node->'Plans' IS NOT NULL AND node->>'Node Type' != 'Bitmap Heap Scan' LOOP
+ node := node->'Plans'->0;
+ END LOOP;
+
+ RETURN COALESCE((node->>'Exact Heap Blocks')::int, 0) > 0;
+END;
+$$ LANGUAGE plpgsql;
+SELECT check_parallel_bitmap_heap_scan() AS parallel_bitmap_instrumentation;
+ parallel_bitmap_instrumentation
+---------------------------------
+ t
+(1 row)
+
+DROP FUNCTION check_parallel_bitmap_heap_scan;
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ebdab42604b..bac97522053 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -188,3 +188,34 @@ select explain_filter('explain (analyze,buffers off,costs off) select sum(n) ove
-- Test tuplestore storage usage in Window aggregate (memory and disk case, final result is disk)
select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
reset work_mem;
+
+-- Test parallel bitmap heap scan reports per-worker heap block stats.
+CREATE FUNCTION check_parallel_bitmap_heap_scan() RETURNS boolean AS $$
+DECLARE
+ plan_json json;
+ node json;
+BEGIN
+ SET LOCAL enable_seqscan = off;
+ SET LOCAL enable_indexscan = off;
+ SET LOCAL parallel_setup_cost = 0;
+ SET LOCAL parallel_tuple_cost = 0;
+ SET LOCAL min_parallel_table_scan_size = 0;
+ SET LOCAL min_parallel_index_scan_size = 0;
+ SET LOCAL max_parallel_workers_per_gather = 2;
+ SET LOCAL parallel_leader_participation = off;
+
+ EXECUTE 'EXPLAIN (ANALYZE, BUFFERS, COSTS OFF, FORMAT JSON)
+ SELECT count(*) FROM tenk1 WHERE hundred > 1' INTO plan_json;
+
+ node := plan_json->0->'Plan';
+ WHILE node->'Plans' IS NOT NULL AND node->>'Node Type' != 'Bitmap Heap Scan' LOOP
+ node := node->'Plans'->0;
+ END LOOP;
+
+ RETURN COALESCE((node->>'Exact Heap Blocks')::int, 0) > 0;
+END;
+$$ LANGUAGE plpgsql;
+
+SELECT check_parallel_bitmap_heap_scan() AS parallel_bitmap_instrumentation;
+
+DROP FUNCTION check_parallel_bitmap_heap_scan;
--
2.47.1
[application/x-patch] v1-0002-Add-regression-test-coverage-for-EXPLAIN-of-Paral.patch (3.9K, 3-v1-0002-Add-regression-test-coverage-for-EXPLAIN-of-Paral.patch)
download | inline diff:
From 7f39a58e9e845709b897276e1b86384bd7f60c3a Mon Sep 17 00:00:00 2001
From: Lukas Fittl <[email protected]>
Date: Sun, 5 Apr 2026 03:48:22 -0700
Subject: [PATCH v1 2/2] Add regression test coverage for EXPLAIN of Parallel
Index Only Scans
The functions dealing with copying back parallel worker instrumentation
such as ExecIndexOnlyScanRetrieveInstrumentation were not exercised
at all in the regression tests, leading to a gap in coverage. Add a
query that verifies we correctly copy back "Index Searches" for
EXPLAIN ANALYZE of a Parallel Index Only Scan.
Reported-by: Andres Freund <[email protected]>
Author: Lukas Fittl <[email protected]>
Discussion:
---
src/test/regress/expected/explain.out | 34 +++++++++++++++++++++++++++
src/test/regress/sql/explain.sql | 32 +++++++++++++++++++++++++
2 files changed, 66 insertions(+)
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 58c5a512d74..b307e810ca5 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -855,3 +855,37 @@ SELECT check_parallel_bitmap_heap_scan() AS parallel_bitmap_instrumentation;
(1 row)
DROP FUNCTION check_parallel_bitmap_heap_scan;
+-- Test parallel index-only scan reports per-worker index search stats.
+CREATE FUNCTION check_parallel_indexonly_scan() RETURNS boolean AS $$
+DECLARE
+ plan_json json;
+ node json;
+BEGIN
+ SET LOCAL enable_seqscan = off;
+ SET LOCAL enable_bitmapscan = off;
+ SET LOCAL parallel_setup_cost = 0;
+ SET LOCAL parallel_tuple_cost = 0;
+ SET LOCAL min_parallel_index_scan_size = 0;
+ SET LOCAL min_parallel_table_scan_size = 0;
+ SET LOCAL max_parallel_workers_per_gather = 2;
+ SET LOCAL parallel_leader_participation = off;
+
+ EXECUTE 'EXPLAIN (ANALYZE, BUFFERS, COSTS OFF, FORMAT JSON)
+ SELECT count(*) FROM tenk1 WHERE thousand > 95' INTO plan_json;
+
+ -- Drill down to the Index Only Scan node
+ node := plan_json->0->'Plan';
+ WHILE node->'Plans' IS NOT NULL AND node->>'Node Type' != 'Index Only Scan' LOOP
+ node := node->'Plans'->0;
+ END LOOP;
+
+ RETURN COALESCE((node->>'Index Searches')::int, 0) > 0;
+END;
+$$ LANGUAGE plpgsql;
+SELECT check_parallel_indexonly_scan() AS parallel_indexonly_instrumentation;
+ parallel_indexonly_instrumentation
+------------------------------------
+ t
+(1 row)
+
+DROP FUNCTION check_parallel_indexonly_scan;
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index bac97522053..3a13fa6ca69 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -219,3 +219,35 @@ $$ LANGUAGE plpgsql;
SELECT check_parallel_bitmap_heap_scan() AS parallel_bitmap_instrumentation;
DROP FUNCTION check_parallel_bitmap_heap_scan;
+
+-- Test parallel index-only scan reports per-worker index search stats.
+CREATE FUNCTION check_parallel_indexonly_scan() RETURNS boolean AS $$
+DECLARE
+ plan_json json;
+ node json;
+BEGIN
+ SET LOCAL enable_seqscan = off;
+ SET LOCAL enable_bitmapscan = off;
+ SET LOCAL parallel_setup_cost = 0;
+ SET LOCAL parallel_tuple_cost = 0;
+ SET LOCAL min_parallel_index_scan_size = 0;
+ SET LOCAL min_parallel_table_scan_size = 0;
+ SET LOCAL max_parallel_workers_per_gather = 2;
+ SET LOCAL parallel_leader_participation = off;
+
+ EXECUTE 'EXPLAIN (ANALYZE, BUFFERS, COSTS OFF, FORMAT JSON)
+ SELECT count(*) FROM tenk1 WHERE thousand > 95' INTO plan_json;
+
+ -- Drill down to the Index Only Scan node
+ node := plan_json->0->'Plan';
+ WHILE node->'Plans' IS NOT NULL AND node->>'Node Type' != 'Index Only Scan' LOOP
+ node := node->'Plans'->0;
+ END LOOP;
+
+ RETURN COALESCE((node->>'Index Searches')::int, 0) > 0;
+END;
+$$ LANGUAGE plpgsql;
+
+SELECT check_parallel_indexonly_scan() AS parallel_indexonly_instrumentation;
+
+DROP FUNCTION check_parallel_indexonly_scan;
--
2.47.1
^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: Fix Heap Blocks accumulation for Parallel Bitmap Heap Scan
2026-04-05 18:51 Fix Heap Blocks accumulation for Parallel Bitmap Heap Scan Lukas Fittl <[email protected]>
@ 2026-04-05 19:08 ` Melanie Plageman <[email protected]>
0 siblings, 0 replies; 2+ messages in thread
From: Melanie Plageman @ 2026-04-05 19:08 UTC (permalink / raw)
To: Lukas Fittl <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>; Tomas Vondra <[email protected]>; Andres Freund <[email protected]>; David Rowley <[email protected]>
On Sun, Apr 5, 2026 at 2:52 PM Lukas Fittl <[email protected]> wrote:
>
> Parallel Bitmap Heap Scans were missing such handling in
> show_tidbitmap_info, causing the information shown on the plan to only
> reflect the Heap Blocks of the leader, not that of the parallel
> workers. I think this is inconsistent, and should be fixed.
I think this was intentional. See David's comment here [1].
- Melanie
[1] https://www.postgresql.org/message-id/CAApHDvqFtd-9DYH70sbjD7iB-Eq-xSip1LPr%3DnayfpPd1pkZVw%40mail.g...
^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2026-04-05 19:08 UTC | newest]
Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-04-05 18:51 Fix Heap Blocks accumulation for Parallel Bitmap Heap Scan Lukas Fittl <[email protected]>
2026-04-05 19:08 ` Melanie Plageman <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox